4. Performance

Frameworks and metrics

Introduction

4.1This chapter discusses the frameworks in place for the measurement and reporting of performance, and how audited entities have approached establishing and developing performance measures that are meaningful to their regulatory responsibilities; setting appropriate targets for performance measures; and reporting their performance based on performance measures and targets according to requirements and advice.

4.2The Public Governance, Performance and Accountability Rule 2014 (PGPA Rule) contains requirements for the performance measures of Commonwealth entities.[1] These requirements are that performance measures must:

• directly relate to one or more of an entity’s purposes or key activities
• use sources of information and methodologies that are reliable and verifiable
• provide an unbiased basis for the measurement and assessment of an entity’s performance
• where reasonably practicable, comprise a mix of qualitative and quantitative measures
• where appropriate, include measures of the entity’s outputs, efficiency and effectiveness
• provide a basis for assessment of an entity’s performance over time.
  3. The PGPA Rule is administered by the Department of Finance (Finance), which also provides advice for implementing the requirements in resource management guides. The Australian National Audit Office (ANAO) has developed a product through its insight series that provides advice to entities on developing and reporting on meaningful performance measures.
  4. Table 4.1 summarises the audit findings with regard to each of issues discussed in this chapter.

Source: Australian National Audit Office

Policy guidance—Resource management guide

4.5The advice contained in RMG 128: Regulator performance (RMG 128) provides information on performance expectations and sets out the three principles of best practice that regulators are required to report against:

• continuous improvement and building trust
• risk based and data driven
• collaboration and engagement.
  6. RMG 128 also outlines the requirements for performance reporting under the Public Governance, Performance and Accountability Act 2013 (PGPA Act) and the PGPARule. Finance advises that regulator performance reporting should be reported through the corporate plan and annual report to support the transparency and accountability of regulator performance.[15]
  7. RMG 131: Developing performance measures (RMG 131) contains information on the concepts for developing performance measures and the specific requirements under the PGPA Rule. Information is presented under the following categories:
• performance measures, purposes and/or key activities
• reliable and verifiable
• unbiased measurement and assessment
• qualitative and quantitative performance measures
• measures of outputs, efficiency and effectiveness
• basis for assessment over time
• tools, techniques and considerations for developing performance measures.[16]
  8. The guide states that it does not offer ‘definitive technical advice on how to design performance measures’ or list generic, standard performance measures.[17]
  9. In its submission to this inquiry, Finance stated that it ‘will also continue to review and update RMGs to feature contemporary examples for entities and regulators to refer to, ensuring our guidance remains fit-for-purpose’.[18] RMG 131 contains only examples from the 2022–23 corporate plans of various entities.[19]

Audit Office insights

4.10In its Insights product, Using performance information to drive effectiveness, the ANAO focuses on the role of accountable authorities, stating that cultural change within entities to embed performance information requires leadership from the top. The observation is made that ‘what gets measured gets done’.[20]

4.11In its Insights product Reporting meaningful performance information, the ANAO identifies six areas of focus within the Commonwealth Performance Framework:

• build a performance focus
• define purposes and key activities
• design meaningful performance measures and targets
• maintain proper records
• report clearly
• improve governance.[21]
  12. During the inquiry, the ANAO further commented on the need for entities to establish meaningful performance measures, stating:

… there's a real challenge for regulators to report on what they're doing and make it meaningful, rather than, 'We did 20 phone calls and 60 letters,' and that kind of thing. It's, 'What impact does the regulation have in meeting the intent of the regulation that they're implementing?' … there's a real challenge here and it's a link between the planning, the activity and the reporting, and what we're seeing is people counting what they do, not taking it towards the regulated impact.[22]

Performance reporting

4.13As outlined in Table 4.1, four of the five ANAO audits under consideration in this inquiry discussed performance measures under the Commonwealth Performance Framework for the regulatory activities examined. The audit of the ATO’s management and oversight of fraud control arrangements for the goods and services tax did not cover performance reporting against the Commonwealth Performance Framework.

Department of Health—Therapeutic Goods Act non-compliance

External performance reporting

4.14The audit report states that the Department of Health and Aged Care’s (Health) corporate plans for 2021–22 and 2022–23 contained four key activities under Program 1.8 (Health Protection, Emergency Response and Regulation) relating to the regulation of therapeutic goods:

• 2021–22:

• regulating therapeutic goods, including vaccines, to ensure safety, efficacy, performance, and quality; promote best practice, monitor compliance, and take appropriate action to address non-compliance
• improving access to therapeutic goods for consumers and streamlining regulatory processes for industry
• delivering efficient, best practice therapeutic goods regulatory outcomes through regulatory science excellence, international collaboration and reform in accordance with the Regulatory Science Strategy2020–2025
• undertaking a range of education activities to inform the public and health professionals on reforms to the regulation of prescription opioid medicines

• 2022–23:

• regulating therapeutic goods, including vaccines, to ensure safety, efficacy, performance, and quality; promote best practice, monitor compliance, and take appropriate action to address non-compliance
• improving access to therapeutic goods for consumers and streamlining regulatory processes for industry
• delivering efficient, best practice therapeutic goods regulatory outcomes through regulatory science excellence, international collaboration and reform in accordance with the Regulatory Science Strategy2020–2025
• regulating nicotine liquid (vaping) products, including education, compliance, and a 2022 review of this regulation.[23]

Established performance measures

4.15The ANAO stated in its audit report that the following key activities from 2021–22 and 2022–23 were relevant to managing compliance:

• 2021–22

• regulating therapeutic goods, including vaccines, to ensure safety, efficacy, performance, and quality; promote best practice, monitor compliance, and take appropriate action to address non-compliance

• 2022–23

• regulating therapeutic goods, including vaccines, to ensure safety, efficacy, performance, and quality; promote best practice, monitor compliance, and take appropriate action to address non-compliance
• regulating nicotine liquid (vaping) products, including education, compliance, and a 2022 review of this regulation.[24]
  16. Of these activities, none had associated performance measures.[25]

Established targets for performance measures

4.17The ANAO found that of the key activities for 2021–22 and 2022–23 covering compliance management listed in the previous section, none had targets in place.[26]

Performance measures and targets are meaningful, relevant and appropriate

4.18The ANAO found that ‘there is little external information on the performance of the department in achieving the expected outcomes of its compliance approach for therapeutic goods’[27] and made the following recommendation to Health:

The Department of Health and Aged Care review its annually reported performance information for its regulation of therapeutic goods to ensure the information is appropriate and covers the significant components of its key activities.[28]

4.19Health agreed to the recommendation. In its submission to the inquiry, Health advised that it is taking the following action in response to the ANAO’s findings:

The department is currently reviewing its external performance reporting in response to the ANAO 2023/24 performance statements audit. This response will also reflect the related recommendation in the Auditor-General Report No.3 2023–24. To support this, the TGA [Therapeutic Goods Administration] is examining how current data is captured and reported, and ways to improve this to enhance insights and measure performance.[29]

Department of Health and ACQSC—Aged Care Reforms

External performance reporting

4.20The ANAO audit report stated that Health mapped care minutes and 24/7 nursing to Program 3.2 (Aged Care Services) of its 2023–‍24 Portfolio Budget Statements.[30] The relevant key activity is ‘Respect, Care and Dignity for older Australians’.[31] Health reported a result of ‘not achieved’ in its 2023–24 Annual Report.[32]

Established performance measures

4.21Under the key activity outlined above, the ANAO identified that Health had developed one relevant performance measure: ‘Older Australians receive residential care services that contributes to their quality of life as measured through: a. provider metrics b. care minutes c. 24/7 nursing’.[33]

Established targets for performance measures

4.22The ANAO audit report noted that the target for Health’s performance measure for aged care reforms was as follows:

a. Establish measurement baseline for ‘Quality of Life’ indicator

b. Maintain average of 200 care minutes per resident per day, including a minimum of 40 minutes of registered nurse (RN) time per day

c. All non-exempt residential aged care facilities have an RN onsite and on duty 100% of the time.[34]

Performance measures and targets are meaningful, relevant and appropriate

4.23The audit did not include an assessment of the appropriateness of the performance measures or targets.

4.24In its submission to the inquiry, Health outlined how it is responding to the audit’s recommendations and stated that:

The department is currently reviewing its external performance reporting in response to the ANAO 2023/24 performance statements audit. This is anticipated to include performance reporting for residential aged care.[35]

Home Affairs—Regulation of migration agents

External performance reporting

4.25In the audit report, the ANAO stated that the Department of Home Affairs (HomeAffairs) had included the Office of the Migration Agents Registration Authority (OMARA) in Outcome 2, Activity 2.1 (immigration and humanitarian programs) and had reported on performance in Home Affairs’ annual report.[36]

Established performance measures

4.26As described below, the ANAO stated in its report that Home Affairs had established targets under Outcome 2, Activity 2.1. These targets are linked to Measure 2.1.1: Effective design, delivery and assurance of immigration programs.[37]

Established targets for performance measures

4.27The ANAO noted in its report that targets were set for regulatory performance for the regulation of migration agents in 2022–23 and 2023–24 as follows:

• 2022–23: ‘100percent of proven instances of non-compliance results in disciplinary action taken’[38]
• 2023–24: ‘75percent of less serious complaints received are resolved within 90days and 50percent of serious complaints received are resolved within 180days’.[39]

Performance measures and targets are meaningful, relevant and appropriate

4.28The ANAO stated that the departmental performance measure for 2022–‍23, ‘100 per cent of proven instances of non-compliance results in disciplinary action taken’, was reported as having been met ‘on the basis that “appropriate” action was taken with the registration of three agents cancelled and one former agent barred for two yearsfrom re-registering’. The ANAO further stated that ‘the reporting did not include instances where agents had been found to be in breach of the Code of Conduct but sanction actions were not taken’.[40]

4.29In its report, the ANAO made the following comments on the quality of Home Affairs’ performance measures:

• ‘there is a risk that the department’s new performance measure for2023–24 will result in complaints being dismissed more quickly rather than greater investigation of complaints that are received’[41]
• ‘there was no performance measure in any year relating to the administration of the registration process for agents or the regulation of [continuing professional development]’.[42]
  30. In its submission to the inquiry, Home Affairs advised that it was reviewing its performance reporting framework in response to the audit.[43] This was supplemented by comments at the public hearing where Home Affairs stated that ‘one of the primary comments in the report was that we could have been better, in terms of reflecting our performance through our regulatory outcomes, so we've implemented four new performance measures for this year’.[44]
  31. Home Affairs further advised in response to questions on notice from the Committee that the four new performance measures for OMARA in Home Affairs’ 2024–25 Corporate Plan are:

• 50% of all investigations (commenced after 1 July 2024) will be finalised within 180 days.
• Reduce the legacy caseload of investigations into Registered Migration Agents by 50%.
• 75% of all early resolution matters will be finalised in 90 days.
• 95% of all initial and repeat applications to become a Registered Migration Agent are assessed within eight weeks of lodgment.[45]
  32. In addition to the content presented on performance reporting against the Commonwealth Performance Framework, the ANAO found that OMARA did not have internal performance measures in place enabling it to measure its performance in processing and managing complaints.[46]

Department of Industry, Science and Resources—Trade measurement

External performance reporting

4.33The ANAO noted in its report that the Department of Industry, Science and Resources (DISR) specified three performance criteria in its 2019–20 Corporate Plan that were linked to the strategic objective ‘Provide a trusted national system of measurement’.[47] The ANAO found that performance under this objective against trader audit targets was not reported in subsequent annual reports, though DISR continued to report internally on this measure.[48]

Established performance measures

4.34The audit report stated that DISR established three performance criteria against the strategic objective ‘Provide a trusted national system of measurement’ in its 2019–20 Corporate Plan.[49] One of the criteria, ‘Percentage of (trade measurement) trader audit target met’, related to DISR’s trade measurement compliance activities.[50]

Established targets for performance measures

4.35The ANAO stated in its report that the target for the performance measure ‘Percentage of (trade measurement) trader audit target met’ was ‘≥90%’ for 2019–‍20.[51]

Performance measures and targets are meaningful, relevant and appropriate

4.36In addition to its findings on DISR’s reporting against the Commonwealth Performance Framework, the ANAO found that DISR had not established targets and reporting arrangements covering its effectiveness as a regulator or progress towards achieving desired regulatory outcomes.[52]

4.37The ANAO also commented on the quality of performance measures listed in DISR’s Regulator Performance Framework Self-Assessment Report for 2019–20, finding that some measures were not meaningful and benchmarks were not established for one measure.[53]

4.38In its submission to the inquiry, DISR advised that it had responded to the audit’s findings by introducing a new performance measure in its 2024–25 Portfolio Budget Statement: ‘National Measurement Institute engagement with traders increases compliance with fair measure regulations’.[54]

2023–24 performance statements audits

4.39The ANAO was funded in the 2021–22 Budget to establish an ongoing program of performance statements audits.[55] This complements the ANAO’s existing assurance function for financial reporting. The ANAO highlights the importance of performance statements auditing, stating that ‘improving links between financial and non-financial performance information is necessary for measuring and assessing public sector productivity’.[56]

4.40In its report on the outcomes of the 2023–24 performance statements audits, the ANAO observed that ‘entities continue to improve their strategic planning and performance reporting processes and practices’ and that there was ‘broad improvement across each of the five categories the ANAO considers when assessing the performance reporting maturity of entities’.[57] The five categories the ANAO considers are: leadership and culture, governance, data and systems, capability, and reporting and records.[58] Nevertheless, the ANAO found that the results from the performance statements audits were ‘mixed’.[59]

4.41All entities that were the subject of audits considered by this inquiry were audited during the ANAO’s 2023–24 performance statements audit program. Overall results for each entity are included in Table 4.2. The ANAO assesses the maturity level of entities using a ‘maturity dial’ with five levels: establishing, developing, baseline, embedded, and advanced.[60] ‘Establishing’ is the lowest level and ‘Advanced’ is the highest level. Definitions of the types of findings made by the ANAO are included as footnotes against the relevant terms.

Source: Australian National Audit Office

4.42Of the findings made by the ANAO, several could be considered relevant to discussion in this chapter and the entities’ performance reporting in general and are listed below.

Department of Health and Aged Care

4.43As noted earlier in this chapter, Health reports on aged care minutes and 24/7 nursing against Program 3.2. The ANAO found that ‘the performance information reported against programs 3.2 and 3.3 was not considered a complete representation of all key functions of the programs. In addition, the ANAO was unable to determine if the results of a number of performance measures were accurately reported as the data was provided by external parties and had not been assured by [Health]’.[72]

4.44The ANAO assessed that 26 of Health’s 34 performance measures were not reliable or verifiable in the annual performance statements.[73]

Home Affairs

4.45The ANAO found:

• Home Affairs lacked appropriate processes to ensure that the data it generated and used for performance reporting was accurate and complete[74]
• when designing targets, Home Affairs should consider the scope and limits of its roles and responsibilities and ensure that reporting accurately states outcomes [that] can be attributed to it[75]
• Home Affairs needed to explain any changes to its targets, methods or the intended presentation of prior year results.[76]

Department of Industry, Science and Resources

4.46The ANAO made a specific moderate finding in relation to performance measure 17 on the National Measurement Institute (NMI). The finding was on ineffective implementation and data quality issues, with the comment that ‘the NMI, within DISR, needed to use adequate risk assessments in selecting initial trader audits and that any data quality issues identified in [sic] are monitored, actioned and followed up’.[77]

Australian Taxation Office

4.47The ANAO found significant responsibilities of the Australian Taxation Office with no performance measure or key activity defined.[78]

Footnotes

[1]Public Governance, Performance and Accountability Rule 2014, section 16EA.

[2]Australian National Audit Office (ANAO), Management of Non-Compliance with the Therapeutic Goods Act 1989 for Unapproved Therapeutic Goods, Auditor-General Report No. 3 2023–24, hereafter TGA Report, paragraph 2.34.

[3]ANAO, Department of Home Affairs’ Regulation of Migration Agents, Auditor-General Report No. 26 2023–24, hereafter Migration agents report, paragraph 1.3.

[4]ANAO, Trade Measurement Compliance Activities, Auditor-General Report No. 5, 2023–24, hereafter Trade measurement report, paragraphs 3.54–3.55.

[5]ANAO, TGA report, table 2.7

[6]ANAO, Design and Early Implementation of Residential Aged Care Reforms, Auditor-General Report No. 8 2023–24, hereafter Aged care report, paragraph 4.43.

[7]ANAO, Migration agents report, paragraph 1.3.

[8]ANAO, Trade measurement report, paragraphs 3.54–3.55.

[9]ANAO, TGA report, table 2.7.

[10]ANAO, Aged care report, paragraph 4.44.

[11]ANAO, Migration agents report, paragraph 1.3.

[12]ANAO, Trade measurement report, paragraphs 3.54–3.55.

[13]ANAO, TGA report, table 2.7.

[14]ANAO, Migration agents report, paragraph 1.4.

[15]Department of Finance (Finance), RMG 128: Regulator Performance, July 2023, hereafter RMG 128, https://www.finance.gov.au/government/managing-commonwealth-resources/regulator-performance-rmg-128, viewed 27 February 2025.

[16]Finance, RMG 131: Developing performance measures, May 2024, hereafter RMG 131, https://www.finance.gov.au/government/managing-commonwealth-resources/developing-performance-measures-rmg-131, viewed 5 March 2025.

[17]Finance, RMG 131, viewed 5 March 2025.

[18]Finance, Submission 2.1, p. 5.

[19]For example, see Finance, RMG 131 – Measures of outputs, efficiency & effectiveness, https://www.finance.gov.au/government/managing-commonwealth-resources/developing-performance-measures-rmg-131/measures-outputs-efficiency-effectiveness, viewed 11 March 2025.

[20]ANAO, Insights: Using Performance Information to Drive Effectiveness, November 2023, hereafter Using Performance Information to Drive Effectiveness, www.anao.gov.au/work/insights/using-performance-information-to-drive-effectiveness, viewed 25 February 2025.

[21]ANAO, Insights: Reporting Meaningful Performance Information, June 2023, hereafter Reporting meaningful performance information, www.anao.gov.au/work/insights/reporting-meaningful-performance-information, viewed 26 February 2025.

[22]Ms Rona Mellor PSM, Deputy Auditor-General, ANAO, Committee Hansard, Canberra, 22 November 2024, p. 41.

[23]ANAO, TGA report, table 2.7.

[24]ANAO, TGA report, table 2.7.

[25]ANAO, TGA report, paragraph 2.34, table 2.7.

[26]ANAO, TGA report, paragraph 2.34, table 2.7.

[27]ANAO, TGA report, paragraph 14.

[28]ANAO, TGA report, paragraph 2.35.

[29]Department of Health and Aged Care (Health), Submission 5, p. 5.

[30]ANAO, Aged care report, paragraph 4.43.

[31]Health, Corporate Plan 2023–24, p. 92.

[32]Health, Annual Report 2023–24, p. 77.

[33]ANAO, Aged care report, paragraph 4.43.

[34]ANAO, Aged care report, paragraph 4.44.

[35]Health, Submission 5, p. 7.

[36]ANAO, Migration agents report, paragraph 1.3; Department of Home Affairs (Home Affairs), Corporate Plan 2023–24, p. 46.

[37]Home Affairs, Corporate Plan 2023–24, pages 46–47. Refer to target 18.

[38]ANAO, Migration agents report, paragraph 1.3.

[39]ANAO, Migration agents report, paragraph 1.3.

[40]ANAO, Migration agents report, paragraph 1.3.

[41]ANAO, Migration agents report, paragraph 3.44.

[42]ANAO, Migration agents report, paragraph 1.4.

[43]Home Affairs, Submission 1, p. 7.

[44]Ms Emily Winch, Senior Director, Home Affairs, Committee Hansard, Canberra, 22 November 2024, p. 35.

[45]Home Affairs, Submission 1.1, p. [10].

[46]ANAO, Migration agents report, paragraph 3.69.

[47]ANAO, Trade measurement report, paragraph 3.54.

[48]ANAO, Trade measurement report, paragraphs 3.54–3.56.

[49]ANAO, Trade measurement report, paragraph 3.54.

[50]ANAO, Trade measurement report, paragraph 3.54.

[51]ANAO, Trade measurement report, paragraph 3.54.

[52]ANAO, Trade measurement report, paragraph 3.79–3.80.

[53]ANAO, Trade measurement report, table 3.8.

[54]Department of Industry, Science and Resources (DISR), Submission 6, pages 9–10.

[55]ANAO, Performance Statements Auditing in the Commonwealth — Outcomes from the 2023–24 Audit Program, Auditor-General Report No. 25 2024–25, hereafter 2023–24 Performance Statements report, paragraph 1.14.

[56]ANAO, 2023–24 Performance Statements report, paragraph 29.

[57]ANAO, 2023–24 Performance Statements report, paragraphs 1.18–1.19.

[58]ANAO, 2023–24 Performance Statements report, paragraph 1.19.

[59]ANAO, 2023–24 Performance Statements report, paragraph 2.7.

[60]ANAO, 2023–24 Performance Statements report, paragraph 3.3, figure 3.1.

[61]ANAO, 2023–24 Performance Statements report, p. 83.

[62]According to the ANAO, ‘A qualified audit conclusion is the most common type of modified audit conclusion and means that the performance statements met the ANAO’s audit criteria, except for one or more areas where the performance statements either did not materially comply with the audit criteria or, due to a lack of evidence available from the auditee, the ANAO was unable to determine if the information presented was materially correct’. See ANAO, 2023–24 Performance Statements report, paragraph 2.9.

[63]According to the ANAO, ‘an Emphasis of Matter paragraph draws a reader’s attention to a matter in the performance statements that, in the auditor’s judgement, is important for readers to consider when interpreting the performance statements’. See ANAO, 2023–24 Performance Statements report, paragraph13.

[64]According to the ANAO, significant (A) findings are ‘findings that pose a significant risk to the entity’s performance statements preparation; these include findings that could result in material misstatement of the entity’s performance statements’. See ANAO, 2023–24 Performance Statements report, Appendix 4.

[65]According to the ANAO, moderate (B) findings are ‘findings that pose moderate risk to the entity’s performance statements preparation; these may include prior year findings that have not been satisfactorily addressed’. See ANAO, 2023–24 Performance Statements report, Appendix 4; ANAO, 2023–24 Performance Statements report, pages 82–85.

[66]ANAO, 2023–24 Performance Statements report, p. 87.

[67]ANAO, 2023–24 Performance Statements report, pages 86–88.

[68]ANAO, 2023–24 Performance Statements report, p. 90.

[69]ANAO, 2023–24 Performance Statements report, pages 89–91.

[70]ANAO, 2023–24 Performance Statements report, p. 71.

[71]ANAO, 2023–24 Performance Statements report, pages 70–72.

[72]ANAO, 2023–24 Performance Statements report, p. 84.

[73]ANAO, 2023–24 Performance Statements report, p. 84.

[74]ANAO, 2023–24 Performance Statements report, p. 88.

[75]ANAO, 2023–24 Performance Statements report, p. 88.

[76]ANAO, 2023–24 Performance Statements report, p. 88.

[77]ANAO, 2023–24 Performance Statements report, p. 91.

[78]ANAO, 2023–24 Performance Statements report, p. 72.