3.1
Chapter 3 sets out the findings of the Joint Committee of Public Accounts and Audit (JCPAA) inquiry into the Commonwealth performance framework, based on Audit Report No. 58 (2016-17), Implementation of the Annual Performance Statements Requirements 2015-16. The objective of the audit was to assess Commonwealth entity progress in implementing the annual performance statements requirement under the Public Governance, Performance and Accountability Act 2013 (PGPA Act) and PGPA Rule 2014.
3.2
The Australian Federal Police (AFP) and the Department of Agriculture and Water Resources (Agriculture) were the audited agencies. AFP and Agriculture were selected for the first audit of the annual performance statements as they demonstrated better practice that might assist other Commonwealth entities.
3.3
The Australian National Audit Office (ANAO) reviewed one purpose statement from these agencies’ 2015-16 Annual Performance Statements and all performance criteria established to demonstrate progress against the following strategic objectives:
AFP—Federal policing and national security
Agriculture—Building successful primary industries
3.4
The audit scope also included reviewing the role of the Department of Finance (Finance) in administration of the annual performance statements requirements.
Committee conclusions and recommendations
Committee conclusions and recommendations
3.6
The Committee makes a number of recommendations on related matters in this chapter and Chapter 2. The Independent Review of the PGPA Act, in progress at the time of the Committee tabling its report, will also cover a range of matters relevant to the Committee’s inquiry. The Committee has made some recommendations for the attention of the review, but the Committee’s primary focus will be to monitor implementation of the review recommendations by Finance and other agencies, noting also that the review’s comprehensive terms of reference were developed in consultation with the JCPAA.
3.7
Improving the Commonwealth performance framework—and, in particular, the quality of performance information to focus on outcomes and strengthen accountability—has been a long-term focus of the JCPAA. A number of significant points related to this matter were discussed at the Committee’s public hearing—in particular, the need:
to use a mix of quantitative and qualitative performance information, along with relevant contextual information and analysis, to focus on entity impacts and outcomes (reflecting the move away from key performance indicators based solely on measuring inputs and outputs)
for narrative utilised as part of qualitative performance information to be evidence-based, reliable and robust
for further work on measurement methodologies for qualitative performance information, drawing on local and international research and practice in this area
for further collaborative work on measuring and articulating performance outcomes, to build consistency and maximise reporting efficiencies
for methodologically robust attribution of entity activities to outcomes that makes accountabilities clear
3.8
The Committee makes a number of recommendations below to address these matters.
3.9
The Committee notes the better practice across most areas demonstrated by the two audited agencies, AFP and Agriculture, in implementing the requirements for the first annual performance statements under the PGPA Act and PGPA Rule. These examples of better practice will assist other Commonwealth entities in implementing the requirements, and the Committee appreciates the commitment by AFP and Agriculture to continuous improvement. Both entities met the minimum preparation and publication requirements for the annual performance statements.
3.10
The Committee commends the ANAO for its development of an effective audit methodology to assess the relevance, reliability and completeness of performance information, and combined prior work by the ANAO and Finance in developing these criteria.
3.11
The Committee also commends Finance for its effective support to entities on the annual performance statements requirements through a range of activities, including Communities of Practice groups, ‘Lessons learned’ publications and updated guidance. These consultative mechanisms are fundamental to driving improvements under the Commonwealth performance framework. The Committee notes that Finance guidance is included in the terms of reference for the Independent Review of the PGPA Act.
3.12
The Commonwealth performance framework aims to ensure a clear read of performance information across corporate plans, Portfolio Budget Statements (PBSs), annual reports and annual performance statements, to improve line of sight between the use of public resources and the outcomes achieved by Commonwealth entities. The Committee notes the ANAO finding that, although the information published in corporate plans and PBSs provides a foundation for reporting in the annual performance statements, there is scope to improve how material is presented to achieve a clearer ‘line of sight’ across performance reporting documents. The terminology across these documents could also be made more consistent (refer paragraph 3.54).
3.13
The Committee recommends that the Australian National Audit Office consider conducting an audit of one complete Commonwealth performance reporting cycle, including whether a clear read of performance information has effectively been established, with consistent terminology and improved line of sight across performance reporting documentation.
3.14
The Committee heeds the timely warning of the Auditor-General that ‘past experience demonstrates leaving external review to periodic performance audits is unlikely to drive the desired level of improvement. This in turn may result in the current reform agenda for performance reporting going the same way as previous ones, with modest improvement and ongoing frustration of the parliament with the quality of performance reporting by entities’.
3.15
The Committee is strongly of the view that, to build on momentum in the implementation of the Commonwealth performance framework, the provisions of the PGPA Act need to be amended to require the Auditor-General to conduct annual audits of performance statements. Mandatory audits will provide the necessary incentive in the system to ensure the quality of that reporting is of the required standard. The Parliament and the Australian public would then receive the same assurance on non-financial performance reporting as on financial reporting, where an independent audit is mandatory. (Mandatory audits will also address some of the matters raised above by the Committee.)
3.16
The Committee agrees that the growing maturity of Commonwealth entities’ annual financial statements can be attributed to the regular external audit scrutiny applied by the ANAO, and that the introduction of mandatory annual audits of performance statements could be expected to lead to similar improvements in the maturity of entity performance statements.
3.17
The Committee supports the Auditor-General’s position that, in the interim, the ANAO should continue to build on its audit methodology in this area such that the ANAO is positioned to be able to audit the annual performance statements of Commonwealth entities in a similar way to the audit of financial statements, when required to do so. Pending this requirement, the ANAO would continue to consider entities’ implementation of the PGPA Act through its annual work program.
3.18
In terms of the requirements in moving to a mandatory system of annual audits of performance statements, the Committee acknowledges the Auditor-General’s observation that to ‘move towards a mandatory system similar to financial auditing would take a number of years, for a number of reasons’:
Firstly, you would want to provide agencies with time to get the quality of their underlying systems and processes into place where an external audit review wouldn’t just be, effectively, qualifying everything that happens because of lack of assurance. So I think an appropriate approach would be to implement such a framework in a gradual way in order to build up the underlying competence in the systems and processes across the sector in order to allow that to happen. I think there are obviously resourcing issues around moving to such a framework, which would need to have consideration as well, and probably ones around capability across the sector. All of those issues would mean that such a move shouldn’t be implemented other than through a transition process.
3.19
Noting the Committee’s legislative oversight role, the ANAO may wish to consider providing the Committee with a briefing on resource implications and its broad preferred timeline for implementation of mandatory annual audits of annual performance statements.
3.20
In terms of building the skills base in this area, the Committee notes that financial reporting falls largely under one professional stream (accounting) but that non-financial performance reporting is more complex in this regard.
3.21
The Committee recommends that:
the Australian Government amend the Public Governance, Performance and Accountability Act 2013 (PGPA Act), and the accompanying rules and guidance as required, as a matter of priority, to enable mandatory annual audits of performance statements by the Auditor-General of entities selected by the Auditor-General for review, with the Department of Finance (Finance) to report back to the Committee on progress on this matter, including consultation with the Auditor-General and Commonwealth entities on implementation timeframes and capacity building
Finance note that the Committee also refers the above matter to the attention of the Independent Review of the PGPA Act
3.22
The Committee recommends that the Australian Public Service Commission (APSC), in consultation with the Department of Finance, the Australian National Audit Office and the Department of Education and Training, conduct a review on whether non-financial performance reporting and evaluation as a training and research discipline requires strengthening, with the APSC to report back to the Committee on progress on this matter.
3.23
The Committee recommends that the Department of Finance, in consultation with the Australian Public Service Commission and the Australian National Audit Office, jointly develop Commonwealth capacity training for non-financial performance reporting and evaluation, as a parallel for existing capacity training for accountancy/financial reporting, and report back to Committee on this matter.
3.24
The Committee regards audit committees as providing Commonwealth entities with an invaluable source of independent assurance and advice. The Committee emphasises that all entities need to give further consideration to the role and functions of their audit committee to ensure that the requirements of the Commonwealth performance framework are met, noting the ANAO’s finding for Agriculture and AFP that neither audit committee could fully demonstrate compliance with the PGPA Rule in terms of reviewing the appropriateness of the annual performance statements. In its response to the ANAO report, Agriculture disagreed with this finding. However, the Committee notes that Agriculture has now agreed to implement the finding, following Finance’s clarification of the audit committee role. The Committee welcomes Agriculture’s concession on this matter. In addition, audit committee charters did not highlight that there must be a mix of relevant skills and experience, including in performance measurement and reporting, within the committee. The Committee emphasises that an audit committee’s charter, and any certification by the audit committee discharging its performance reporting function, should reflect this requirement.
3.25
The Committee recommends that:
the Australian Government amend, as necessary, the Public Governance, Performance and Accountability Act 2013 (PGPA Act), and accompanying rules and guidance, to clarify that the functions and charter of Commonwealth entity audit committees need to reflect their role in assurance of the appropriateness of performance reporting, as well as specifying that some members must have skills in performance measurement and reporting, with the Department of Finance (Finance) to report back to the Committee on progress on this matter
Finance note that the Committee also refers the above matter to the attention of the Independent Review of the PGPA Act
3.26
The Committee notes that AFP and Agriculture had established or adapted existing systems and processes to meet the requirements of the PGPA Act and the PGPA Rule for the annual performance statements. The Committee also notes that the majority of results presented in the annual performance statements of AFP and Agriculture were supported by complete and accurate records, as required by the PGPA Act and PGPA Rule.
3.27
The ANAO made no recommendations in its audit but identified a range of matters that warranted further attention by AFP and Agriculture, as well as some key learnings of relevance to all Commonwealth entities in preparing their annual performance statements. Both entities provided an update on their implementation progress regarding the audit findings, and the Committee acknowledges their progress in this regard.
3.28
The Committee again commends the ANAO and Finance for their work in identifying comprehensive ‘key learnings’, ‘opportunities for improvement’ and ‘lessons learnt’ from each corporate plan and annual performance statements cycle, providing entities with an invaluable reference source. The Committee points to the usefulness of a consolidated reference to such material, to assist with further embedding these findings and drive continuous improvement.
Review of evidence
3.29
This section reviews the evidence received by the Committee regarding:
Measurement and reporting of performance, and ‘line of sight’
Mandatory annual audits of annual performance statements
Supporting systems, assurance and record keeping
Finance support and guidance
Key learnings for all Commonwealth entities and implementation of audit findings
PGPA Act requirements
3.30
Under the Commonwealth performance framework, established under the PGPA Act and PGPA Rule, Commonwealth entities are required to publish corporate plans each reporting period. Annual performance statements, included in the annual report, provide an assessment of an entity’s progress in achieving its purposes, as set out in its corporate plan and aligned to the PBS. The publication of the corporate plan represents the beginning of a performance cycle and the annual performance statements the end of a cycle. Corporate plans are intended to be the primary planning documents of Commonwealth entities. The first corporate plans were required to be published by 31 August 2015 (unless otherwise specified), and the first annual performance statements were required to be included in entities’ 2015-16 annual reports.
3.31
The ANAO concluded that AFP and Agriculture ‘met the minimum requirements’ for the preparation and publication of the first annual performance statements under the PGPA Act and PGPA Rule. Both entities’ annual performance statements were published as part of their annual reports, and they included the ‘required statements, results and analysis against the performance criteria outlined in the corporate plan and Portfolio Budget Statements reviewed as part of the audit’. Matters to be included in annual performance statements are set out at Box 3.1.
3.32
In terms of the ‘Results’ section of the annual performance statements, the ANAO noted that both entities addressed the requirement to provide the results of the measurement and assessment of their performance in achieving their purposes. The performance statements included results against each performance criterion and references to the location of each criterion within the entities’ PBSs and corporate plans. However, Agriculture’s 2015-19 Corporate Plan ‘did not include information on the data collection techniques that would be used to measure results’. It was noted that Agriculture advised the ANAO of changes it had made to address this matter.
3.33
The ANAO further noted that the ‘Analysis’ section of both entities’ annual performance statements ‘included some consideration of the entities’ operating environment and were supported in some cases by case studies and trend information’. Both entities ‘included some analysis of the factors contributing to the reported performance’, in accordance with the requirements of the PGPA Rule. However, the ANAO found that the ‘quality of the analysis could be improved, in particular, by providing further discussion of how the entities’ activities, through the results of the performance criteria, had contributed towards the achievement of their purpose/s and the external factors which impacted performance’. In response, the AFP noted that ‘more detailed analysis for both individual performance criteria … and overall will be provided in the APS 2016-17 to assist the reader in assessing performance’.
Box 3.1
Under section 16F of the PGPA Rule, Commonwealth entity annual performance statements must:
measure and assess the entity’s performance in achieving the entity’s purposes in the reporting period in accordance with the method of measuring and assessing the entity’s performance in the reporting period that was set out in the entity’s corporate plan, and in any Portfolio Budget Statement, Portfolio Additional Estimates Statement or other portfolio estimates statement, that were prepared for the reporting period; and
include the following information in the annual performance statements:
a statement that the performance statements are prepared for paragraph 39(1)(a) of the Act;
a statement specifying the reporting period for which the performance statements are prepared;
a statement that, in the opinion of the accountable authority of the entity, the performance statements: accurately present the entity’s performance in the reporting period; and comply with subsection 39(2) of the Act.
The results of the measurement and assessment referred to in subsection (1) of this section of the entity’s performance in the reporting period in achieving its purposes.
An analysis of the factors that may have contributed to the entity’s performance in achieving its purposes in the reporting period, including any changes to: the entity’s purposes, activities or organisational capability; or the environment in which the entity operated; that may have had a significant impact on the entity’s performance in the reporting period.
Measurement and reporting of performance, and line of sight
3.34
The ANAO has developed an audit methodology to review the relevance, reliability and completeness of Commonwealth entity performance information—see Figure 3.1.
3.35
In terms of the first criterion of relevance, the ANAO concluded that the performance criteria in the annual performance statements were ‘mostly relevant’ to the activities undertaken by AFP and Agriculture, but ‘alignment of entity activities to performance criteria and measurement of the attribution of specific activities to the achievement of the entities’ purposes could be enhanced’. Specifically:
One of the AFP’s performance criteria required improvements to assist the reader to identify the benefit or beneficiary measured by the performance criterion and its link to the AFP’s activities.
Agriculture’s performance criteria were mostly relevant, however the benefit or beneficiary was often not clear, or the focus of the measure was not clearly attributable to the entity.
3.36
To address the above finding, AFP noted that the result for this performance criterion had been ‘augmented with more detail about the survey (e.g. who the respondents are) and multiple case studies of joint operations to show how strong stakeholder relationships are integral to interoperability and successful case outcomes’. Similarly, Agriculture explained how it was addressing the above finding in preparing for its next annual performance statements, which would ‘include more detailed analysis and contextual information outlining the activities that we undertook versus things that weren’t undertaken by the department. That should help others understand the overall delivery environment in the context of what that performance measure was being delivered in and whether or not that’s met our strategic objective’.
3.37
In terms of the second criterion of reliability, the ANAO concluded that both entities’ performance criteria ‘mostly provided a reliable method of assessing the entities’ progress in fulfilling their purposes’, but addressing any ‘potential bias’ in the reported results needed to be considered. The entities also need to describe the ‘methodology for measurement and basis for assessment, including through a target or baseline’, to improve the reliability of their performance criteria. Specifically:
Improvements to two of the AFP’s performance criteria are required to limit the level of potential bias in the reported results.
The majority of Agriculture’s performance criteria did not describe the method or basis for measurement, or provide a target or baseline, impacting the reliability of the performance criteria.
3.38
As to the challenges in addressing potential bias in reported results, the Auditor-General emphasised the need to establish ‘appropriate evaluation frameworks early in the development of a program’:
the most effective way to combat bias with respect to evaluation is to make it as external to the agency as possible. As a starting point, with respect to evaluating programs, ensure that the evaluation framework is established before you start the evaluation. An important element is to set up the indicators of success at the beginning, not when you start to do the review work, because that will remove some of the bias from it. This includes clearly establishing the data and their sources on which you’ll base your assessment, whether it be qualitative or quantitative, so that you can reduce bias in the processes by setting those issues upfront. That’s something that we comment on quite regularly in our audit reports around establishing appropriate evaluation frameworks early in the development of a program.
3.39
Mr Ramzi Jabbour, Deputy Commissioner Capability, AFP, further reflected on this matter, pointing to some of AFP’s ongoing challenges in measuring and articulating performance outcomes:
We’ve been trying to grapple with this for about six years. We have consulted internationally and we have consulted academic institutions here in Australia and overseas, economists and financial folk, in trying to consider the impact and effect, particularly in the disruption-prevention space. What is the value or return on investment in relation to the effort we’ve applied to an investigation? How do you weight one criminal versus another? At the moment, if we look at our current reporting and the quantitative data, one prosecution is equal to another prosecution. That’s not necessarily the case, because one may be a kingpin, which has a significant impact and effect on the disruptive activity that we’ve caused in relation to a syndicate that is impacting Australia’s national security, versus a low-level courier. At the moment, if you look at effective prosecutions, one equals one …
The other challenge is: in the criminal environment, when you topple one there are many others ready to take their place. Again, it can be a little bit challenging. We can certainly talk about the value of an individual or a value proposition, but how effective that is in disrupting the activities may not be seen for some time. It may take six to 12 months to realise the full impact that a particular investigation or measure has in relation to that particular crime type …
We have enhanced the 2017-18 corporate plan based on the feedback that was quite instructive through the audit. We’re certainly more focused in this corporate plan on the activity and the purpose. The purpose is clearly articulated that our strategic direction is exactly that—to place greatest effort where we believe we are going to achieve the most significant impact and effect. Measuring that, however, is the challenge for us. Certainly our direction, our intent—where we are going to invest most heavily—is absolutely clear, but, as I say, trying to articulate the outcome and the effectiveness of that is particularly challenging.
3.40
AFP further commented that a ‘narrative’ might provide an effective option here:
This is a space that we are particularly interested in, but there is no easy solution, quite frankly. We’re heartened, certainly, by the work that’s being done and we’re very, very eager to develop more quantitative measures. But from our research to date, consistent with what has been said today, there is no country in the world that we’re aware of that has mastered this. It may be that we’re over-engineering it. Maybe it’s a narrative; maybe, it’s as simple as that. Maybe it’s a narrative that says: the intelligence reveals that this individual … has been involved in this level of criminality. If we have consensus through the intelligence community that that’s right, it might be as simple as that, and we understand the impact.
3.41
Measuring and articulating outcomes, and using a combination of qualitative and quantitative performance information to focus on outcomes and not just outputs, continued to be a matter of some interest throughout the public hearing. As to whether there was tolerance for ‘narrative’ being incorporated into the assessment of performance, as referred to above, the ANAO confirmed that ‘it’s a requirement in the framework at the moment’. As Finance further elaborated, ‘that’s very much the main focus of our framework—that mixture of qualitative and quantitative measurement … narrative does play a very important role … because it describes the numbers ... if people provide a bit of analysis and a narrative behind that, as long as it’s supported by evidence, that gives a richer performance story because it tells you the full story, not an interpretative story’.
3.42
As the Auditor-General emphasised, ‘the issue, as we move into that type of framework, is making sure that the narrative is evidence based’—for example, ‘use of case studies is seen as a good way of providing narrative. The Audit Office would agree that that’s a good way. But, if the case study used is an anomaly from the policy, then it actually isn’t helping tell the story of what’s going on. You need the narrative to be demonstrably evidence based as well, rather than assertion based’.
3.43
As to whether entities were using evidence based narrative, underpinned by methodological robustness, and a mixture of qualitative and quantitative performance information in their annual performance statements, AFP responded:
Not as effectively as … we could. We do have a narrative; we do have case studies. But we need to look at doing exactly as has been described here, quite frankly—having the intelligence or evidence base to underpin the narrative around the significance of the investigation and to try to articulate that in terms of the impact and effect that the application of the resources has had to the outcome … there is scope for us to improve the way we do that at the moment—without doubt.
3.44
Similarly, Agriculture responded that it was ‘looking to also enhance the level of reporting in that respect. In this year’s performance statements, we will see a greater level of narrative and the use of case studies. But, importantly, we are looking to make sure there is analysis and contextual information as part of that so it links back into the performance objective’.
3.45
As to the rigour around qualitative and quantitative performance information, Finance observed that ‘measuring inputs and outputs is pretty easy. Measuring efficiency is pretty easy … Outcomes and impacts are more difficult to measure’:
There are issues that go to attribution of activities to outcomes. There are issues around the measurement methodologies and the role of proxy measures and so on. Not only is it tough stuff but it really gets to the point where accountability has become quite stark. So people are a little bit worried about it … There is a bit of pushback in the system in relation to this because it makes accountabilities clear.
3.46
The ANAO noted that ‘attribution’ of activities to outcomes is a key element here:
The new framework goes a long way towards allowing entities to design a fit-for-purpose performance statement which uses all of the elements that are available to them as far as going all the way back to the inputs that they are using, the outputs that they are generating, the outcomes and impacts where they can measure them in both qualitative and quantitative senses, as well as providing the contextual information which is now a requirement in this framework to actually explain in writing what that all means and how that all comes together … that is the key element of the framework—that actual piece of analysis that pulls the whole result together and allows entities to explain themselves.
3.47
Regarding some of the challenges of trying to measure outcomes, the Auditor-General emphasised the ‘value of this particular framework is that it provides a greater degree of capacity around that because it requires the agency, in the performance statements, to provide not only quantitative measures of what it’s doing but also a qualitative analysis of achieving its purpose’, and ‘where you have qualitative analysis, you need to be assured that the underlying assertions in that are reliable and robust’.
3.48
In terms of the third criterion of completeness, the ANAO concluded that the performance criteria included by AFP and Agriculture in their annual performance statements were ‘substantially complete, collectively providing a balanced basis for assessing the entities’ progress in fulfilling their purposes.’ However, the ‘selection of performance criteria will require ongoing effort by both entities to identify opportunities to clarify or increase the overall alignment of performance criteria to the purpose’. The entities’ ‘balance of performance criteria—for example, qualitative, quantitative, efficiency-focused and short, medium and long term timeframes—should be reviewed’. (This matter was discussed above.)
3.49
There was also interest at the public hearing in understanding more about the approach used to assess annual performance statements. The Auditor-General explained that there are two levels to the way the assessment is undertaken:
One is the high-level criteria around relevance, reliability and completeness type issues. That type of framework is reasonably robust. The framework we used drew largely on work that Finance has done and that we’d done in the past on what an appropriate framework assessment was. You’d expect that to mature over time. Underlying that, the ANAO developed an audit methodology to implement on how to undertake the audit process around assessing those criteria. To develop that methodology, we looked at other jurisdictions who had been undertaking such work, such as Western Australia and New Zealand, who had audit methodologies in those processes, and adapted that to the Australian Commonwealth experience.
3.50
Finance observed that it was ‘working closely with the ANAO to put in place the key elements of the framework, including the guidance and clarity around the standards we expect in performance information’, and ‘the relevance, reliability and completeness framework is one that appears to be working. It’s clear to understand. It’s broken down into subcategories or requirements, where, if people aspired to meet them, they would see a significant improvement in the quality of all performance information that’s reported to the parliament across all of the different reporting methodologies’. However, Finance noted that the methodology for qualitative performance information was ‘less well developed’:
In some areas, like key performance indicators, there are well-established criteria and benchmarks for what makes for a good KPI. In more qualitative methodologies and evaluations in benchmarking studies and whatever, it’s a bit more loose, or less well developed. There’s not much in international practice that helps you to nail down stronger frameworks in that area. So we’re doing a bit of blue-skying in this area.
3.51
Mr Lembit Suur, First Assistant Secretary, Finance, pointed to the international experience with this area:
Internationally, everybody is struggling with this stuff. New Zealand … [has] 10 key performance areas and proxy measures in relation to some of them to show the success of their policies. In the United Kingdom and some other countries work is happening on attribution in particular because the really difficult stuff which has to do with intergenerational impacts and so on is quite vexed. Anyone working in that space is taking a risk because they don’t have much else to rely on. I chair an OECD network on performance and results, and one of the things I am trying to do is push that network’s work next year into looking at measurement methodologies—in particular, outcome measurement and impact measurement. I think, as a community, all OECD countries could benefit in having a conversation in this space. It might unlock some good ideas.
3.52
Mr Suur further noted that there is ‘an emerging community of practice profession in this space’—for example, an Australian Evaluation Society conference recently brought together some 400 people from across Australia and New Zealand, from within government and the evaluation industry and academe, to talk about evaluation as a discipline, but ‘the question is how to tap and harness it and get it doing stuff that leads to reporting to the parliament about what’s being achieved with public funds’.
3.53
There was also interest in ways to maximise the efficiency of the annual performance statements process in terms of time and resources. AFP emphasised that further collaborative work on outcomes reporting would be beneficial:
In the AFP we have nine people in this area, dedicated to performance reporting and analysis. It’s both: it’s not just producing the report but also analysing and collecting the data. There are some areas that we are looking to enhance, particularly our IT systems, which don’t necessarily capture the level of data that we would like as easily as we would like. So it is quite resource intensive, but there’s a significant investment required in that, and that is being undertaken as part of an IT enhancement program over a number of years. The one area … we would all benefit from … is in relation to trying to better understand how we can report on outcomes … a collaborative effort in relation to that is the way to go and would save us all time and effort.
3.54
The Commonwealth performance framework aims to ensure a clear read of performance information across corporate plans, PBSs, annual reports and annual performance statements (with information also being consistent and comparable across entities and reporting cycles), to improve line of sight between the use of public resources and the outcomes achieved by Commonwealth entities.
3.55
The corporate plan sets out the performance criteria (how performance will be measured and reported) that the annual performance statements report against. The corporate plan generally sets out a ‘Purpose’, a ‘Strategic Objective’, ‘Activities’ and ‘Performance Criteria’. The annual performance statements generally set out a ‘Purpose’ (also aligned to an ‘Outcome’ in the PBS, and a ‘Strategic Objective’/’Program’ in the corporate plan and PBS), ‘Results’ (against the ‘Performance Criteria’ in the corporate plan and PBS) and ‘Analysis’.
3.56
The ANAO examined performance criteria established by AFP and Agriculture in their 2015-19 Corporate Plan and how their 2015-16 Annual Performance Statements reported against those criteria. The ANAO concluded that the ‘information published by the entities in their corporate plan and Portfolio Budget Statements provides a foundation for reporting in the annual performance statements, although there was scope for both entities to improve how material was presented to achieve a clearer “line of sight”’.
3.57
More specifically, the ANAO found that the AFP 2015-19 Corporate Plan ‘met the requirements of the PGPA Rule and provided a foundation for performance reporting’. However, the ANAO noted that the presentation and expression of AFP’s activities could be ‘improved to assist a reader to identify and assess the alignment of the performance criteria to the purpose’, as it could be ‘difficult for the reader to easily identify specific activities, and obtain a clear read of how the activities contribute towards the AFP’s purpose and objectives’. Regarding this finding, the ANAO noted that AFP had identified the presentation and expression of its activities as an area for improvement as part of its ongoing review of the performance measurement and reporting framework. As a result, the AFP’s 2016-17 Corporate Plan has been structured so that activities intended to be undertaken by AFP to fulfil its purpose are ‘clearly articulated’ and ‘provide the basis for a clearer read between the 2016-7 Corporate Plan and the performance statements to be included in the 2016-17 Annual Report’.
3.58
Similarly, the ANAO found that the Agriculture 2015-19 Corporate Plan ‘met the requirements of the PGPA Rule and provided a foundation for the reporting of Agriculture’s annual performance. The key priorities and objectives published in the 2015-16 PBS were reflected in the 2015-19 Corporate Plan purpose statement, meeting the requirements of the PGPA Act’. The plan also had ‘clear links to the outcomes and programs within the PBS. The PBS performance measures and targets are a subset of the performance measures included in the 2015-19 Corporate Plan, consistent with Finance guidance’. However, the ANAO noted that there was ‘scope to better focus the purpose statement, to allow the intended results to be more readily identifiable’ and ‘improvements could be made to the identification of key activities by including the specific activities that the department will undertake’. Regarding this finding, the ANAO noted that the ‘structure of the purpose statement was improved in the 2016-17 Corporate Plan’.
3.59
There was interest at the public hearing in further exploring how to get a clear read on outcomes, as much as outputs, across performance reporting documentation. The Auditor-General emphasised that the Commonwealth performance framework seeks to embed an ‘outcome focused framework’ that provides the Parliament with a ‘clear read’ across performance reporting documents:
what we have seen over time with respect to performance frameworks is that, across most of the countries that we compare ourselves to, there has been a movement trying to go from what was 30 or 40 years ago an input based approach through looking at outputs into more outcome focused frameworks through time. I think what we are trying to see embedded in the Commonwealth with the current performance framework is something that gives the parliament a clear read of how resources are being used, from the economy with which inputs are procured through to the efficiency with which outputs are delivered and the impact of those outputs on achieving the results, outcomes, purposes of the entity. That is quite a mature framework … I would say that the way that is being defined in the Commonwealth is if not best practice then pretty close to it from what I have observed around the place.
3.60
However, as discussed below, the Auditor-General pointed to the importance of having ‘incentives in the system’, in the form of mandatory annual audits of performance statements, to ‘ensure that the quality of that reporting is of the standard necessary to meet the parliament’s purposes’.
Mandatory annual audits of annual performance statements
3.61
The Auditor-General’s responsibilities, as set out in the Auditor-General Act 1997, include auditing the annual performance statements of Commonwealth entities in accordance with the PGPA Act. However, as the Auditor-General noted at the public hearing, the ‘PGPA Act does not require the Auditor-General to conduct audits of performance statements unless requested by either the Minister for Finance or the responsible minister. This means the parliament does not receive assurance as a matter of course, on performance statements, as it does over financial statements, where an independent audit is mandatory’.
3.62
As the Auditor-General emphasised, ‘the growing maturity of Commonwealth entities’ annual financial statements can in part be attributed to the regular external audit scrutiny applied by the ANAO’, and the ‘introduction of annual audits of performance statements could be expected to lead to similar improvements to the maturity of entities’ performance measurement and reporting’:
Engagement with entities throughout a financial year provides the opportunity to resolve matters affecting the reliability of financial statements in real time. This approach, accompanied by the ANAO’s bi-annual reporting of significant and moderate audit issues to the parliament, has played a role in entities moving towards the more mature financial reporting processes observed today.
3.63
The Auditor-General further observed that ‘past experience demonstrates leaving external review to periodic performance audits is unlikely to drive the desired level of improvement. This in turn may result in the current reform agenda for performance reporting going the same way as previous ones, with modest improvement and ongoing frustration of the parliament with the quality of performance reporting by entities’.
3.64
In particular, the Auditor-General pointed to mandatory annual audits of performance statements as providing an ‘incentive’ in the system to ensure that the ‘quality of that reporting is of the standard necessary to meet the parliament’s purposes’:
I think the way the framework is set up for the presentation of corporate plans through to performance statements should provide the basis for quality reporting on outputs and outcomes. The one proviso I’d put to that is: what are the incentives for that to occur? The most important thing is not only that the parliament gets a report but that there are incentives in the system to ensure that the quality of that reporting is of the standard necessary to meet the parliament’s purposes … which goes back to my comments about external review.
3.65
As the Auditor-General further emphasised, ‘I think we need to look at the incentives, because we’ve had a number of frameworks over a number of years which haven’t quite met expectations when they started out, and it’s not certain that, if we keep doing the same thing, we’ll get a different result’:
the parallel for me is to the external review that we undertake on financial statements. It is an incentive in the system for the quality and accuracy of reporting. I think that performance reporting is as important in the public sector as financial reporting …
The committee has seen the impact of reporting on things like the MPR, the Major projects report, where, over a decade, you’ve seen the improvement of reporting in that product as a result of not just what we do but the combination of the incentives we put into the system and the work we do, along with the external reporting and transparency that that increases in that context.
3.66
The Auditor-General confirmed that, in the interim, ‘we will continue to build on our audit methodology in this area and intend to position the ANAO to be able to audit the annual performance statements of Commonwealth entities in a similar way to the audit of financial statements, if required to do so. Pending such a requirement, the ANAO will continue to consider entities’ implementation of the PGPA Act through our annual work program’.
3.67
On whether mandatory audits of annual performance statements would drive a culture that would report as much on performance outcomes as outputs, the Auditor-General confirmed that the framework ‘already sets out the requirement to link what the agency does to the purpose of the agency, and the purpose of the agency is embedded in the outcomes that it wants to achieve … What it comes to then is the overarching quality of what is coming up … around relevance, reliability, completeness and those types of issues’.
3.68
As to whether, if mandatory audits were implemented, the ANAO would push for qualitative as much as quantitative data, the Auditor-General confirmed that is the ‘nature of the framework. The framework sets out that that should happen. What the parliament wants to be assured of, I would believe, is that the nature of those qualitative statements [is] linked to evidence, in fact, and can be relied upon … It’s not going to be perfect, but the question is whether it’s better than not having it in the first place’.
3.69
In terms of what would be required to implement this framework, the Auditor-General responded that to ‘move towards a mandatory system similar to financial auditing would take a number of years, for a number of reasons’:
Firstly, you would want to provide agencies with time to get the quality of their underlying systems and processes into place where an external audit review wouldn’t just be, effectively, qualifying everything that happens because of lack of assurance. So I think an appropriate approach would be to implement such a framework in a gradual way in order to build up the underlying competence in the systems and processes across the sector in order to allow that to happen. I think there are obviously resourcing issues around moving to such a framework, which would need to have consideration as well, and probably ones around capability across the sector. All of those issues would mean that such a move shouldn’t be implemented other than through a transition process.
3.70
There was also interest at the public hearing in establishing what other jurisdictions, both within Australia and internationally, had external review of non-financial performance reporting. The Auditor-General responded that ‘from the external review perspective, Western Australia has had external review of performance information for a couple of decades now. New Zealand has had it for a while; I can’t recall how long. I’m not aware how much formal review goes on across the rest of the world’. In terms of Western Australia, the Auditor-General further noted that, ‘my understanding is that, in the nineties, they introduced a system where the Audit Office undertakes a review of the performance information that the agencies produce and looks at its appropriateness and provides an assurance review report to the parliament on the appropriateness of the performance information’.
3.71
Finance observed that ‘practice is mixed’ in this regard:
Different people have gone down different paths. The structure of reporting, who the reporting goes to and what is done as reporting varies from jurisdiction to jurisdiction. But … in terms of conceiving at a whole-of-system level what non-financial reporting might look like for accountability purposes to the legislature, we’re probably as advanced as anybody. In fact, some of our thinking is better advanced than it is in many other OECD jurisdictions.
3.72
In terms of the skills base for non-financial reporting, Finance emphasised that financial reporting is the ‘domain of the accountants’—there is one professional stream that looks after financial reporting, but for non-financial performance reporting there is ‘not one domain or stream; this is an area that needs a broad church approach to it. That adds to the complexity because you’re trying to bring in a whole group of people from different professional areas and different perspectives to try to get this done. That is why it is probably a bit more complex than financial reporting’.
3.73
As the ANAO noted on a related point, the certification required by an accountable authority of their performance statements is equivalent to that of the financial statements, but reporting of performance information is not necessarily subject to the same established framework of internal controls or accounting standards that apply in respect of the annual financial statements. As entity non-financial performance reporting frameworks mature, ‘management structures and systems of internal control for performance statements can be expected to mirror those of the financial statements’:
Chief Financial Officers have responsibility for the preparation, and joint certification with the accountable authority, of the financial statements. To meet this responsibility, there is an expectation that the Chief Financial Officer has the relevant knowledge, skills and experience of accounting to prepare the financial statements. Similar expectations were not evident within the performance statements certification processes.
Supporting systems, assurance and record keeping
Supporting systems
3.74
The ANAO concluded that both AFP and Agriculture had ‘established or adapted existing systems and processes to meet the requirements of the PGPA Act and the PGPA Rule’ for the preparation of annual performance statements. These systems and processes ‘remained in development during the audit, with further work being undertaken in 2016-17, to support the quality of information reported in future performance statements’. In addition, ‘both entities developed a project plan outlining the roles and responsibilities, risks and mitigating controls, milestones for delivery, and assurance mechanisms to guide the preparation of the performance statements’.
3.75
However, ‘neither entity completed a comprehensive preassessment of the processes of producing performance statements as part of their planning. The incremental development of performance reporting by both entities was focused on identifying lessons learnt’. The ANAO also found that, while both entities had ‘established, or leveraged from existing systems and methodologies to collect and report performance information for the purposes of the annual performance statements’:
The AFP would benefit from considering the frequency and extent of reviews of the methodology supporting one performance criterion, and strengthening processes for the validation of information sourced outside of management systems.
Further consideration of data availability is required by Agriculture to establish a system to support the consistent collection, analysis and reporting of non-financial performance information.
3.76
As the Auditor-General summarised at the public hearing, both entities would ‘benefit from earlier planning, including consideration of data availability, and implementing a continuous reporting process’. As the Auditor-General further noted, ‘one of the key indicators of the quality of work, of the information being presented, is the quality of the controls underlying the systems that produce the data—the quality assurance that sits over it … and also the extent to which the information isn’t solely developed for one purpose: that is, reporting’:
So, in financial reporting, we’ve found that, over time, as agencies have become more mature in their systems of monthly reporting to executives around financial performance, you get a better line of sight through the year into an end-of-year report with better quality. As you know, the corporate plan is required to be your primary planning document. As the information in that becomes embedded as the core management framework for agencies, flowing through management reporting through a year, and the systems underpinning that become more robust and more repeatable, then you should get better quality information, as long as the control environment is correct. So that goes to the quality of what you’re reporting, not its relevance.
3.77
The ANAO provided further information at the public hearing about its finding regarding Agriculture’s need to improve data availability in support of its performance reporting, noting that ‘it was a lack of depth in the planning, so 12 months on, when the performance statements need to be pulled together, there were some issues in relation to some data sources’:
It wasn’t across the board; it was just in isolated areas, and then it meant that Agriculture needed to move pretty quickly to find some new data sources for their report … The depth of planning before publishing a corporate plan needs to be adequate to provide some serious veracity around the data sources and the information that’s going be provided later on in the performance statement.
3.78
On this finding, Agriculture confirmed that ‘we have made a range of changes in preparing for this year’s performance statements, both in documenting the sources of data and in ensuring that, as we continue to progress, we can align the performance measures and the data that supports those together … we are continuing to review both the performance measures and the data that supports them, and … this year you’ll see much stronger alignment’.
Assurance: Audit Committees
3.79
The PGPA Act specifies that Commonwealth entities must establish an audit committee. Section 17 of the PGPA Rule sets out the functions of an audit committee (which must also be set out in a written charter), as follows:
The functions must include reviewing the appropriateness of the accountable authority’s:
performance reporting; and
system of risk oversight and management; and
system of internal control for the entity.
3.80
The ANAO found that both Agriculture and AFP had ‘established assurance processes to certify that the reported performance information accurately reflected entity performance’. However, the ANAO noted that ‘refinement of these arrangements is required by the entities’, including documenting guidance on the assurance process, and the review and endorsement of the annual performance statements to ensure that evidence of management and audit committee assurance is recorded and retained.
3.81
In particular, the ANAO concluded that both entities should give further consideration to the role and functions of their audit committee to ‘ensure that the intent and requirements of the framework are met, as neither audit committee could fully demonstrate compliance with the PGPA Rule’. The ANAO found that ’there would be benefit in both entities further considering the role of the audit committee as a source of independent assurance to the accountable authority and how their audit committee charters and processes establish a basis to provide this assurance’. In addition, ‘audit committee charters did not highlight that there must be a mix of relevant skills and experience, including performance measurement and reporting, within the committee’. As set out above, the PGPA Rule provides that an audit committee’s functions must include reviewing the appropriateness of the entity’s performance reporting. This function would necessarily involve an audit committee forming a view on how the entity should measure its performance. As a result, an audit committee’s charter, and any certification by the audit committee to an accountable authority discharging their performance reporting function, should reflect this requirement.
3.82
By way of further background on this matter, the ANAO noted that ‘each entity relied on management certifications over the selected performance criteria and the completeness and accuracy of underlying records’, and the entities’ audit committees also received regular briefings on the preparation of the annual performance statements, including details on the management certification processes. In particular:
Agriculture’s audit committee commissioned an internal audit on the Key Performance Indicators (performance criteria) and Performance Reporting, to inform the committee’s review responsibilities. The audit committee also monitored the implementation of key recommendations. However, the audit committee’s sign off to Agriculture’s accountable authority was limited, and did not meet the requirements of the department’s audit committee charter or the PGPA Rule and its intent.
The AFP was unable to locate the final certification by its audit committee to the accountable authority over the performance statements, limiting an assessment against the audit committee charter, or the PGPA Rule and its intent.
3.83
In its written response to the audit, as included in the Audit Report, Agriculture disagreed with the ANAO’s finding on this matter:
The department does not accept the finding that its Audit Committee did not meet the requirements of its charter, or the requirements and intent of the PGPA Rule, in providing assurance of the certification process. The committee undertook a range of work to meet its charter, and the department considers the assurance provided was consistent with advice from the Department of Finance on the role of audit committees in the certification process. The department is committed to establishing processes to ensure the Audit Committee meets the requirements of the PGPA Rule.
3.84
The ANAO responded that:
1. The Audit Committee’s Charter required the committee to review and provide independent advice and assurance about the appropriateness of the department’s performance reporting … The ANAO concluded that the advice provided to the Accountable Authority by the Audit Committee did not provide assurance about the appropriateness of the department's performance reporting …
2. … The Department of Finance has acknowledged the need for further clarity in guidance to audit committees.
3.85
The Auditor-General provided further information on this matter at the public hearing, confirming that the PGPA Rule sets out the same audit committee requirements for annual performance statements as for financial statements, and ‘that’s around providing assurance to the accountable authority on the appropriateness of the statement. What we found in the context of the department of agriculture is that the assurance that was provided to the accountable authority wasn’t consistent with that rule, as it didn’t provide that level of assurance over the performance statements’.
3.86
In explaining its disagreement with this ANAO finding, Agriculture noted ‘we had a different interpretation of that at the time … that specifically related to ensuring that the term “appropriateness” was include[ed] in the guidance to the accountable authority … our position was based on the knowledge and information we had at the time’. Agriculture confirmed that Finance had now provided ‘additional guidance with regard to the way in which that should be structured. We will be following that and changing our position for the coming performance statements to provide additional assurance as outlined in the new guidance’. Agriculture further confirmed that:
The intention this year is to have specific wording that will include reviewing the appropriateness of the accountable authority’s reporting in providing that to the secretary. This year we’ve gone through and changed certification internally on those performance statements. It requires our senior executives not only to document the process they’ve gone through to certify that but also a checklist that requires them to ensure that particular elements, such as the collection of data—where it has come from—is included. That will go through to the audit committee as part of the review, coming back to the secretary.
3.87
The ANAO noted that the need for further clarity in guidance on audit committees had been ‘acknowledged’ by Finance. As Mr Lembit Suur, First Assistant Secretary, Finance, confirmed:
We issued some additional guidance. The rule, which was put in place in 2014, says that an audit committee has to review the appropriateness of performance reporting. One of the clarifications that we issued is that the word ‘appropriate’ has its common English language meaning, which is ‘fitness for purpose’. I don’t see that that’s a breakthrough piece of information that we provided. But we did, following discussions with different people, give more information about how you might go about establishing whether or not information is fit-for-purpose.
3.88
In its submission to the inquiry, Finance also noted that:
Since the finalisation of the audit, Finance has published a paper on the role that Audit Committees play in reviewing the appropriateness of performance information … This paper clarifies the role of audit committees in providing accountable authorities with independent advice and assurance on the appropriateness of their performance reporting. The paper was developed in consultation with key stakeholders. The Finance Secretary has written to all Accountable Authorities to draw their attention to this paper.
3.89
The ANAO noted that the Independent Review of the PGPA Act will provide an opportunity to further consider the Finance guidance.
Record keeping
3.90
Section 37 of the PGPA Act sets out the requirement for Commonwealth entities to keep records regarding its performance reporting, as follows:
The accountable authority of a Commonwealth entity must cause records to be kept that properly record and explain the entity’s performance in achieving its purposes.
The accountable authority must ensure that the records are kept in a way that:
complies with any requirements prescribed by the rules; and
enables the preparation of the annual performance statements required by section 39.
The responsible Minister and the Finance Minister are entitled to full and free access to the records kept under this section. However, those Ministers’ access is subject to any Commonwealth law that prohibits disclosure of particular information.
3.91
The ANAO found that the ‘majority of results’ presented in the annual performance statements of both AFP and Agriculture were ‘supported by complete and accurate records as required by the PGPA Act and PGPA Rule’. However, both entities could ‘improve record-keeping to better demonstrate the calculations and analysis applied to raw data to produce results, and to support the analysis’. Specifically:
The AFP maintained complete and accurate records for all but one measure.
Agriculture was unable to provide complete records for one performance criterion, and relied on an absence of advice as confirmation of compliance for another.
3.92
AFP outlined how it was addressing this finding in its submission to the inquiry.
Finance support and guidance
3.93
The ANAO audit scope included reviewing Finance’s role in whole-of-government administration of the annual performance statements requirements.
3.94
To assist entities in implementing the Commonwealth performance framework, Finance has:
published written guidance in the form of Resource Management Guides (RMGs)
reviewed draft corporate plans and performance statements on behalf of entities, when requested;
established the Performance Community of Practice (Performance CoP) to provide a forum to share expertise and examples of better practice and establish a feedback loop between framework design, implementation and results. Four Performance CoP performance statements workshops were conducted in 2017;
released regular Public Management Reform Agenda newsletters and emails to keep entities informed of emerging better practice and other developments;
published lessons learned papers in February 2016 and January 2017, based on Finance’s assessment of a selection of 2015-19 and 2016-20 corporate plans, including examples of better practice; and
released a draft lessons learned paper in March 2017 and published the final paper in May 2017, based on Finance’s assessment of a selection of 2015-16 performance statements, including examples of better practice.
3.95
The ANAO noted that guidance from Finance is also being ‘incrementally updated as lessons are learnt’. The ANAO observed that the Independent Review of the PGPA Act will provide an opportunity to further consider the Finance guidance.
3.96
As to their participation in Finance’s Community of Practice meetings, Agriculture responded that ‘we see great value in participating in a range of communities of practice. We believe they will continue to help us develop better performance statements into the future as one input into our work there ... We do find it useful and believe it will be really important, moving forward, as we try and progress and build on the work we’re doing in this space’. Similarly, AFP responded that ‘we have been very active in the Department of Finance’s community-of-practice group and all the sessions that they have been running’.
3.97
Finance provided more detail about its Community of Practice meetings, noting that ‘we invite all Commonwealth entities and companies to attend’ and ‘one of the most successful aspects of this framework has been that opportunity for entities to share’—‘we run events in Canberra, and in Sydney and Melbourne as well to try and reach as many entities and companies as we possibly can … This is a broad framework, and it is when you get down to the nuts and bolts of it and have individual conversations that the value of it comes out’. Finance emphasised that this consultative mechanism had been ‘fundamental in driving some of the improvement that we have seen in this framework to date’.
Key learnings for all Commonwealth entities and implementation of audit findings
3.98
The ANAO noted that this is the first year of published annual performance statements under the PGPA Act and ‘it is expected that entity processes will take some time to mature’. On this basis, the ANAO did not make any audit recommendations, but instead highlighted a range of matters that ‘warrant further attention by AFP and Agriculture’, as well as some key learnings that might be considered by other Commonwealth entities in preparing their annual performance statements—see Box 3.2. AFP and Agriculture were selected for the first audit of the annual performance statements as they demonstrated better practice that might assist other Commonwealth entities.
3.99
In its submission to the inquiry, AFP noted that ‘in keeping with previous audit results, the AFP rated reasonably well on all audit criteria’ and had ‘already addressed a majority of the issues highlighted in the audit’. However, AFP observed that it ‘may take several planning and reporting cycles to fully realise the improvements’. To address ongoing challenges in articulating performance outcomes, AFP confirmed that it would focus on ‘continued review and reform. This is facilitated by maintaining a central performance area, active participation in the Department of Finance performance community of practice and also by maintaining a corresponding localised, law enforcement/regulatory agency performance peer group’.
3.100
In its response to the audit, Agriculture noted that its work under the Commonwealth performance framework is an ‘evolving process’ and the department had ‘already made a range of changes to its performance framework’, with the audit providing an ‘opportunity to identify further improvements’. Agriculture also observed that it agreed with ‘most of the ANAO’s findings indicating areas in which its performance measurement and reporting can be improved. These findings will inform its continued efforts’.
Box 3.2
Presentation of results and analysis
Presenting results alongside established targets, and providing comparisons to results from previous years or references to related indicators and results, can assist the reader in assessing performance.
The analysis section of the performance statements provides entities the opportunity to supplement the reported results with contextual information. This can enhance the reader’s understanding of the environment within which the entity operates and the contributions that it makes.
Purposes and activities
Clearly identifying and grouping activities can assist the reader to assess the alignment of the performance criteria with the entity’s purpose. This provides the basis for a clear read between the corporate plan and the performance statements.
Relevance, reliability and completeness of performance criteria
Relevant performance criteria should clearly align to an entity’s purpose and activities, indicate who will benefit from the related activity and how, and be understandable to readers.
Reliable performance criteria should be measurable, disclose the method or basis for assessment such as a target or benchmark, and not lead to biased results.
Complete performance criteria should collectively address the entity’s purpose, and provide a balanced examination of the entity’s effectiveness and efficiency across the different forms of performance information and their timeframes.
Systems, processes and methodologies
The enhanced Commonwealth performance framework provides entities the opportunity to apply a fit for purpose approach to their performance measurement. This principle should be kept in mind by entities in designing or reviewing their own processes to inform the preparation of the annual performance statements.
Entity processes should support the complete cycle of performance measurement and reporting. This would reflect the preparation and publication of the Portfolio Budget Statements, corporate plans, performance statements and annual reports, evaluating lessons learnt and incorporating these into the following cycle to contribute to continuous improvement.
Where appropriate, entities should consider how existing IT systems and controls can be leveraged from to support performance reporting. This would include clearly documenting any calculations or analysis applied to data drawn from systems to enable recalculation.
Assurance processes
A check list is a useful tool to assist entity management representatives to understand the detailed considerations expected, including the level of evidence required, to support a certification of performance criteria and accompanying results.
An audit committee’s charter, and any certification by the audit committee to an accountable authority discharging their performance reporting function, should clearly reflect the PGPA Rule and its intent.
Record-keeping
To ensure appropriate access to supporting records for future years, entities may consider establishing a centralised repository.
Clearly document and retain records detailing the methods used for calculations and analysis to reach the result reported in the annual performance statements to enable recalculation.
3.101
As to how its ‘key learnings’ would be promoted to entities, the Auditor-General responded that, ‘from an Audit Office perspective, our primary engagement with agencies outside performance audits is through audit committees. We attempt to share all the learnings that we get from our audit activity through audit committees into agencies … We have people attend all audit committees and, as a matter of course, they would present a report, which usually includes the findings from various audit reports’.
3.102
Finance observed that the ‘key learnings’ are ‘really useful’ because they go to ‘improving the quality of practice across the Commonwealth. It is the ANAO providing some pretty practical insights about what can be looked at and what can be done to improve the quality of performance information’. Finance further noted that ‘there’s nothing in those that goes to the framework or the framework design. They all go to the practice. So we continue to have community of practice meetings. We continue to update our guidance’.