Performance Audit Report No.17 (2012–13)

Chapter 2 Design and Implementation of the Energy Efficiency Information Grants Program

Introduction

2.1                   The Energy Efficiency Information Grants (EEIG) program is a $40 million competitive merit-based grants program. It was designed to provide funding to industry associations and non-profit organisations to work with small and medium size business enterprises and community organisations to provide information about the smartest ways to use energy, empowering them to make informed decisions about energy efficiency. The program was introduced as part of the Government’s Clean Energy Future Plan.[1]

2.2                   The Department of Climate Change and Energy Efficiency (DCCEE) was responsible for the administration of the EEIG program during the audit period.[2] As part of Machinery of Government changes announced in March 2013, the DCCEE was abolished and its energy efficiency functions transferred to the Department of Resources, Energy and Tourism (RET). RET now has responsibility for the continuing administration of the program.

2.3                   The program’s first funding round was opened to applications on 13 February 2012 and closed on 16 March 2012.[3]

2.4                   Following eligibility assessment by the DCCEE, an independent program advisory committee (PAC) was responsible for assessing each of the 188 eligible applications against three merit criteria, and allocating a merit ranking to each. The department recommended to the Minister for Climate Change and Energy Efficiency the 28 applications that were put forward by the PAC as being the most meritorious, each of which was approved for funding by the Minister in May 2012. The total value of the projects approved in the first round was approximately $20 million.[4]

2.5                   A second EEIG funding round was opened to applications on 30 October 2012, with a closing date of 20 December 2013.[5] At the time of writing, the successful applicants had not been publicly announced. According to the department’s website, applicants were due to be notified of the second round outcome in May 2013.[6]

Audit objective and scope

2.6                   The objective of the Australian National Audit Office (ANAO)’s audit was to assess the effectiveness of the design and implementation of the EEIG program. The focus of the audit was the preparation for, and conduct of, the program’s first funding round.

2.7                   The audit examined the program against relevant policy and legislative requirements for the expenditure of public money and the grants administration framework (including the Commonwealth Grant Guidelines).[7]

Audit conclusion

2.8                   The audit report concluded that the DCCEE ‘was well resourced to design and implement the EEIG, and the design of the program was effective’, but there were ‘significant shortcomings in the conduct of the assessment process for applications’.[8]

2.9                   The ANAO found that program governance arrangements and a range of internal documentation to support the administration of the program were developed, and a robust set of program guidelines were published. The department also planned a sound approach to undertaking application eligibility checks, assessing the merit of eligible applications and ranking them for the Minister’s consideration.[9]

2.10               However, the audit found that the merit assessment process ‘departed in important respects from that outlined in the program guidelines, and inadequate records were made and retained to demonstrate that each application was assessed in accordance with the published eligibility and merit criteria’.[10] The main shortcomings outlined in the report were that:

n  most eligible applications were allocated to one of four merit categories, with each of the 28 applications allocated to the highest merit category (termed ‘outstanding’) then placed into one of six ranking bands (in case the Minister did not wish to approve all 28 ‘outstanding’ applications). Neither process was foreshadowed by the program guidelines. In addition, the allocation of the 28 ‘outstanding’ applications to a ranking band used a process that did not relate to the score each application had achieved in terms of the published merit criteria;

n  DCCEE destroyed records made by each PAC member of the assessment of each eligible application against the three published merit criteria, notwithstanding that the contractual arrangements specified that these were official records and that DCCEE had made no arrangements to otherwise record the scoring by PAC members. The minutes of the PAC meetings were also too brief to provide any insight into the merit assessment and scoring of each eligible application; and

n  fortunately for DCCEE, one PAC member made and retained his own electronic record of some of the scoring results but there is no record, official or otherwise, as to how each of the 28 recommended (and approved) applications had been assessed and scored against the three published merit criteria.[11]

2.11               The DCCEE advised the ANAO that it had confidence in the PAC, had followed probity advice, and had ‘no reason to believe that the successful projects were not the most competitive against the merit criteria’. However, the ANAO concluded that it was ‘not possible to be satisfied that the most meritorious eligible applications … were recommended to the Minister for approval’.[12]

Audit recommendations

2.12               Noting that a second EEIG funding round was underway, the audit report made four recommendations, as shown in Table 2.1.

Table 2.1        ANAO recommendations, Audit Report No.17 2012–13

1.	In circumstances where an advisory panel or committee is used to assess grant applicants and to provide funding recommendations, ANAO recommends that the Department of Climate Change and Energy Efficiency: (a) develop program governance arrangements that inform panel/committee members that they are formally operating under the Commonwealth’s financial framework; and (b) implement secretarial support arrangements that require key assessment decisions and their basis to be minuted. DCCEE response: Agreed
2.	To be able to demonstrate that grant program applicants were treated equitably and in accordance with the published program guidelines, ANAO recommends that the Department of Climate Change and Energy Efficiency maintain adequate records of its assessment of applications in terms of their eligibility and compliance with other mandatory (or gateway) criteria. DCCEE response: Agreed
3.	The ANAO recommends that the Department of Climate Change and Energy Efficiency, in planning for the implementation of competitive, applications‐based grant programs, develop strategies to effectively manage the risk of funding rounds being over‐subscribed including, as appropriate, identifying in the program guidelines any adjustments that may be made to the eligibility and/or merit assessment approach where this situation arises. DCCEE response: Agreed
4.	The ANAO recommends that, consistent with the transparency and public accountability principles of grants administration outlined in the Commonwealth Grant Guidelines, the Department of Climate Change and Energy Efficiency ensures that the assessment and merit ranking of applications and any significant changes as they proceed through the assessment process are appropriately documented. DCCEE response: Agreed

The Committee’s review

2.13               Representatives of the following organisations gave evidence at the Committee’s public hearing on 19 June 2013:

n  Australian National Audit Office

n  Department of Resources, Energy and Tourism.

2.14               The Committee’s evidence covered the following issues:

n  Resourcing of the program

n  Record-keeping deficiencies

n  Improvements in second funding round.

Resourcing of the program

2.15               The EEIG program guidelines stated that the PAC would assess the eligible applications against the three weighted merit criteria—‘project effectiveness’, ‘project design and management’ and ‘value for money’—allocating a merit ranking to each and advising the Minister, through the department, on projects suitable for funding.[13]

2.16               The DCCEE’s planning was based on receiving an estimated 30 to 40 applications for funding. In fact, the department received 207 first round applications, of which 188 were assessed to be eligible. The ANAO found that the ‘impact of the high number of applications received … was particularly evident at the merit assessment stage’ conducted by the PAC.[14]

2.17               The ANAO highlighted that the DCCEE had not developed strategies for how it would respond to receiving significantly more applications than it anticipated.[15] It observed that, given the higher than expected number of applications, the time allotted for the assessment of the EEIG applications was ‘insufficient’ in the context of the resources available. It suggested that possible management strategies that could have been employed included ‘increasing the resources allocated to the merit assessment task and/or extending the planned timeframe’. The ANAO recommended that, in planning future grants programs, the DCCEE develop strategies to effectively manage the risk of funding rounds being oversubscribed.[16]

2.18               In its response to the ANAO’s recommendation, the DCCEE advised that it would ensure ‘sufficient resources are made available for secretariat support to the PAC’.[17]

2.19               At the Committee’s public hearing, RET confirmed that, of the $40 million total funding for the program, approximately $6 million (or 15 per cent) had been allocated to administration. When the Committee questioned why the administration costs were this high, RET explained that:

It is a program that runs for three or four years. There are some start-up costs in terms of assessing that program I think there were some comments made by the auditors in terms of benchmarking that. But it is not just a one-off. We have got to start this and we are actually managing this through. So we will manage that series of projects over the life of the projects.[18]

2.20               Asked about the planned outcomes of the department’s use of the part of this administration funding for ‘ongoing monitoring’, the department explained that there were two elements to this work:

The first is the contract monitoring [by] departmental staff, and most of the contracts have multiple milestones over several years. We also have an independent evaluation which looks at the influence of the program—if you like, the efficacy of the information generated—and whether that has influenced behaviour in the recipient businesses.[19]

2.21               Noting that the funding allocated to the department for administration seemed relatively high, the Committee asked how the deficiencies reported by the Auditor-General came about. The department identified the ‘resourcing of the secretariat function of the merit assessment and the support of the merit assessment committee’ as being one ‘specific element’ contributing to the shortcomings. It noted that the department’s experience of allocating more time and resources to those processes in the second funding round had ‘certainly led to a better product in terms of records taken’.[20]

2.22               The department also confirmed for the Committee that there was no involvement from the Minister’s office in the application assessment process, aside from the initial approval of program guidelines and the final consideration of the department’s recommendations for funding.[21]

Record-keeping deficiencies

2.23               The audit report documented a number of deficiencies in the records kept by the DCCEE of decisions which informed its recommendations to the Minister. These deficiencies included that:

n  Minutes of PAC meetings contained insufficient detail relating to assessments of applications and the presence or absence of members with conflict of interest declarations;[22]

n  Decisions made by departmental officials who conducted initial eligibility assessments were not well-documented;[23]

n  The scoring templates used by PAC members to assess each eligible application were destroyed by the department because they were seen as ‘rough working drafts’;[24] and

n  The department did not seek to capture the final agreed PAC scores for each eligible application against the merit criteria, or the resulting aggregate scores for the 28 applications that were categorised as ‘outstanding’.[25]

2.24               The ANAO summarised its findings in relation to record-keeping as follows:

Some important records made by the PAC members of their initial assessment and scoring of each application were destroyed by DCCEE, and the department did not otherwise document the results of the merit assessment process. Consequently, it is not possible to be satisfied that the published merit criteria were applied in a consistent and robust manner, or for DCCEE to demonstrate that the most meritorious applications were identified and recommended to the Minister for approval.[26]

2.25               The audit report did not include details of how individual scoring templates came to be destroyed by the department. However, at a Senate Estimates hearing in February 2013, the ANAO stated that the destruction was carried out by ‘the more junior staff’. It explained that when the PAC members returned their copies of the applications to the department, the attached scoring sheets had been destroyed along with the applications. The ANAO advised that, by ‘asking a lot of questions to the relevant people involved’, they were satisfied that this destruction was a result of oversight rather than intent. However, the ANAO also advised that this ‘unauthorised destruction of a record’ was a breach of the Archives Act.[27]

2.26               In the Committee’s public hearing, RET agreed unreservedly with the ANAO’s recommendations and stated that record-keeping:

… has been recognised very clearly by the department as an area that needs to be improved, and it has been improved in the subsequent round.[28]

2.27               However, RET advised that it considered the destruction of the initial assessment documents of panel members to be appropriate. The department outlined that the decision had been made by a senior executive officer, based on advice from the probity adviser and a working understanding of what would constitute ‘normal administrative practice’ under the Archives Act 1983. The department added:

The advice that we had at the time from the probity adviser and from our subsequent reading of what is known as normal administrative practice under the Archives Act is that they were working notes. They were working notes of committee members acting as individuals not of the committee as a whole. In that sense, we still do not think that they were records under the definition in the Archives Act.[29]

2.28               The ANAO did not agree with this assessment, stating that the panel members’ contracts described the scoring template as an official record to be returned to the department. The ANAO considered the destruction to be ‘quite a significant matter’ due to the inadequacy of other documentation of the decisions underpinning the final recommendations. The ANAO explained that:

When it comes to grant assessment processes it should be not simply about recording what score is given to each application by the panel members but about the reasoning behind that score so that there can be some accountability and transparency around the basis on which panels provide recommendations to departments and departments provide recommendations to ministers so that we, auditors, parliament and other stakeholders can be confident that the most meritorious applications were identified and assessed on a basis consistent with the program guidelines.[30]

2.29               The ANAO added that it would expect to see this matter addressed in any future audit of the program:

We would certainly suggest that if we were auditing the second round and the department did not require something similar to these templates to be completed in some fashion so that there is a record of what each application was scored against each criteria and the reasoning for that, we would be equally critical of the second round as we were of the first round.[31]

2.30               Responding to these comments, RET agreed that record-keeping should fully justify the decisions made by advisory committees and that this had not adequately occurred in the first round of the program. The department affirmed that it had ‘absolutely’ changed its practices following the audit to improve its records on the deliberations of the committee.[32]

Improvements to second funding round

2.31               During the public hearing, RET updated the Committee on the program’s second funding round. It advised that 175 applications had been received, and 18 applications were successful.[33] In response to a question on notice, the department noted that seven of the successful second round applicants had also applied under the program’s first round.[34]

2.32               In a tabled opening statement, RET informed the Committee that ‘taking the lessons from the first round into the administration of the second round of the program has assisted the department to improve the program’s administration’. The department advised that it had been able to implement all of the responses to the audit findings that it had outlined, with the key actions being:

n  More time for the assessment process which allowed the Department to deal with the over-subscription, while following processes as outlined in the program guidelines.

n  Both better procedural guidance on the secretariat functions and further clarity in the probity briefing on the role and responsibility of the Program Advisory Committee members.

n  Extra staff and clearer procedural guidance on the taking of minutes of the deliberations of the Program Advisory Committee ensured both enhanced transparency and ability to provide feedback to applicants.[35]

2.33               At the public hearing, RET expanded on its implementation of the ANAO recommendations with the following comments:

n  The standard operating procedures and the probity briefing to clarify the relationship with the program advisory committee and to align with the Commonwealth financial frameworks have been put in place, so there is now clarity in terms of the role of the program advisory committee.

n  We have made sure that the merit assessment process includes assessment and commentary in relation to each evaluation criterion and ensured that flows all the way through the documentation, so the documentation is very thorough.

n  A probity adviser now sits on all the governance committees, so the program advisory committee, and is there full-time and has been there full-time in the second round.

n  ...we have updated our standard operating procedures to ensure accurate and adequate record-keeping of completeness and eligibility checks.

n  ...we have ensured that the guidelines provide clarity in relation to the processes around eligibility and merit assessment.

n  We have implemented strategies to ensure that the volume of applications coming in could be catered for in the process we have put in place.[36]

2.34               Reflecting on the broader lessons of the audit in relation to grants program processes, the Auditor-General indicated that the ANAO’s audit coverage was driven by the desire to ensure that all grant applicants are treated equitably. He noted that ‘the pleasing thing about this audit is that the department has not argued the toss at all on this matter and the matters of substance here’.[37]

2.35               The Auditor-General added that the department had administered ‘aspects of this program very well’, but that there were some problems particularly in the assessment area. He voiced his concern that the public sector ‘do not learn the lessons of the prior experience’ as quickly as desirable. However, he expressed optimism that RET had ‘gained from this’ and flagged that the ANAO would ‘continue to emphasise the importance of sound practices in this area’.[38]

Committee Comment

2.36               The Committee supports the ANAO’s findings and recommendations, and agrees with its conclusion that, while aspects of the program were managed well, there were significant administrative shortcomings in the assessment phase.

2.37               It was concerning that the ANAO could not be satisfied that the most meritorious applications had been recommended for approval. However, the Committee is satisfied that the program’s shortcomings resulted from poor administrative practices, rather than any malicious intent.

2.38               From the evidence received, it was clear that RET was well-resourced for the program’s administration. However, the department did not internally allocate enough time or resources to the assessment of applications in the first funding round, significantly contributing to the program’s shortcomings.

2.39               There remains some level of disagreement between the department and the ANAO in relation to whether the individual scoring sheets completed by advisory committee members should be considered official records and formally retained under the Archives Act. However, the Committee observed agreement from both sides on the broader points that: it is essential for program records to document the basis on which recommendations for funding are made; and this had not occurred in the program’s first funding round.

2.40               The results of this audit contain valuable lessons for officials involved in grants programs about the importance of conducting fair and transparent decision-making processes that are supported by appropriate record-keeping. The findings also highlight the importance of sound risk management processes for early identification of strategies for dealing with unexpected events—in this case, a much larger than expected number of applications.

2.41               Encouragingly, the key lessons from the audit appear to have been learned and applied by RET in the second funding round, with more time and resources having been allocated to the assessment of applications. The department has also pointed to a range of measures that it has put in place to improve the integrity and transparency of the program, particularly in relation to documentation.

2.42               The Committee accepts the department’s advice that it has improved its practices, and expects that any future audits of this program, or similar RET grants programs, will contain more positive findings. Any reoccurrence of issues similar to those identified in this report should be viewed as very concerning by the Auditor-General, the JCPAA, the broader Parliament and the public.