External reporting
3.1
The committee examined the reintegration of the former Defence Materiel Organisation (DMO) back into the Department of Defence (Defence), and how Defence publicly reported its expenditure and activities.
Reintegrating DMO into Defence
3.2
The Committee was advised that in 2011, the financial authority for sustainment passed from DMO back to the Capability Manager (the relevant service chief).
3.3
Following the First Principles Review, the decision was made to reintegrate the former DMO, now known as the Capability, Acquisition and Sustainment Group (CASG) back into the Department of Defence. This had positive cultural effects, noted by Vice Admiral Ray Griggs, who stated:
No longer do we hear the phrase ‘Defence and DMO’, and you don’t hear ‘Defence and CASG’. It is one organisation.
3.4
Further, responsibility for deciding how sustainment funding is prioritised and reallocated is also now a matter for the Capability Manager (the appropriate Service Chief). The Deputy Secretary of CASG, Mr Kim Gillis, described how this worked in practice:
I'm very comfortable if I find out about a sustainment issue after the service chief does. That doesn't bother me, because I think their accountability is directly to make sure that they're sustaining the fleet. If we have a fleet-wide issue, regularly I'm the second person who gets the phone call because they're talking directly to the capability manager because he is the customer that we are providing to. I'm not saying that we didn't do that in the past, but we're really emphasising that that accountability for the delivery of the war-fighting capability and the 'raise, train and sustain' sits clearly in the accountability of the capability managers, and we are there to deliver that for them.
3.5
Dr Tom Ioannou of the ANAO stated that there had been ‘positive trends’ in this regard, and more broadly that:
If [recent developments] come together in the way that's expected, I think we'll see progress, but I think—I'll end where I started—we've seen many years of reform activity. Not all of it has been successful. The same themes keep coming up again and again, and I think the real challenge always for Defence is the hard yards of consolidation.
Transparent public reporting to Parliament
3.6
Defence’s sustainment framework reporting encompasses a number of different unclassified reports. This includes:
Portfolio Budget Statements
Portfolio Additional Estimates Statements
3.7
The level of information in the above reports is constrained to that which is publicly releasable, but does comprise expenditure information on the top 30 sustainment products, which is 77 per cent of total sustainment expenditure.
Clear read from budget papers to PBS, PAES, Annual Report
3.8
A ‘clear read’ is an important accountability measure. The term ‘clear read’ means that the objectives for each project should be clearly stated at the start of the year and achievements against those objectives reported at the end of the year. The Department of Finance describes the concept in official guidance:
The ‘clear read’ between corporate plans and Portfolio Budget Statements on the one hand, and annual reports and annual performance statements on the other, is an essential part of the accountability system that compares performance targets and figures against the outcomes actually achieved.
3.9
The ANAO stated the importance of a ‘clear read’:
The quality of pubic reporting is improved where there is a ‘clear read’ between reporting documents and where information is reconciled. The ANAO would expect to receive information from Defence, in its response to the draft audit report, whether applying this approach to current reporting on the Top 30 sustainment products might give rise to security concerns.
3.10
Defence provides an update on project performance and variance explanations (from forecast expenditure) for the top 30 Defence Sustainment products in the Defence Portfolio Budget Statements (PBS), the Defence Portfolio Additional Estimates Statements (PAES) and the Defence Annual Report (DAR). These top 30 projects account for approximately 77 percent of the total sustainment expenditure.
3.11
It is an important principle that all reports should include both financial and descriptive information. The ANAO noted that while the financial reporting for the top 30 sustainment products was adequate, there were variations in descriptive information:
At times, there is no explanation of variations in the full year outcomes. There’s also use of different terminology throughout the documents, and some of the descriptions aren’t linked. Some of the descriptions of what was achieved versus what was planned are not linked to each other.
3.12
Defence agreed that there were ‘some inconsistencies’, and stated that there were inconsistencies in public reporting as a result of a ‘more measured’ approach to public reporting, as opposed to that provided to the Government and Ministers.
3.13
The ANAO replied that Defence may benefit from providing internal structural guidance to clearly articulating how planned and actual performance is reported.
Improving transparent public reporting
3.14
The Committee discussed various models for improved public reporting, from enhancing current processes to developing a sustainment document similar to the Defence Major Projects Report (MPR).
3.15
In considering the merits of an MPR-style document, the ANAO stated:
The consolidation of sustainment information would improve its accessibility and the level of transparency, whether it is published as a separate document or as part of an existing one.
The costs associated with increasing the transparency and depth of information beyond that currently published would depend on the type and availability of the information sought.
3.16
At the public hearing, the Auditor-General further discussed various reporting models and indicated that, as sustainment processes were more mature than acquisition was a decade ago, an MPR style document would be costly and less than optimal.
3.17
The Auditor-General suggested that the ANAO could:
…work with Defence to ensure that some of its existing external reporting is more consistent across the required planning and reporting documents, the PBS, corporate plans and annual reporting. We could put in place, from the ANAO perspective, a set of periodic reviews to look at the macro framework, like this audit—every several years, going in and providing assurance about the progress in Defence's improvement of its sustainment processes, along with a series of specific audits relating to specific projects or specific sustainment activities, similar to what we do at the moment with specific acquisition projects.
So, rather than trying to go broad, going narrow and deep with those strategic overviews on a risk basis might be a more cost-effective model of work that we could do for the Parliament.
Transparent public reporting and national security
3.18
In its submission, Defence outlined its concerns in finding a balance between transparency and not compromising national security by providing too much information about current capability levels:
Of utmost importance is the need to be cognisant of the National Security risks associated with providing a high level of public visibility regarding military capability. Assessments around the readiness and availability of major Defence capabilities are by necessity classified. A recent review by Defence Intelligence Organisation determined that the current public reporting regime is “safe”. In this context any proposal for new reporting requirements will need to consider if the new information might be aggregated to disclose classified information on capability readiness and availability.
3.19
Defence outlined its concerns around national security and public reporting, noting that there would always be ‘limitations on our public reporting around sustainment, because it cuts to the heart of readiness and preparedness’. Further, Defence noted that ‘this is really a core issue for us, and it is going to always have a level of tension between [the Parliament’s] rightful needs and our need to maintain security.’
3.20
The Committee asked Defence how other jurisdictions handled national security sensitivities around sustainment reporting, with Mr Kim Gillis replying:
I've consulted with a number of my counterparts, national armaments directors, and our reporting is actually, in some cases, fuller than anybody else's reporting that I've been able to find. I've tried to find a set of sustainment reports which give indications of preparedness levels, and I can't find a military in the world that's willing to share that information. The major projects report that we have on major acquisitions was taken from our UK cousins, and that's what we modelled it on. I've been working with DE&S [Defence Equipment and Support], which is the counterpart organisation, to do a benchmark study against them.
Line between acquisition and sustainment
3.21
The Committee asked whether Defence had a clear line on where asset acquisition ended and sustainment of an asset began, and which capability issues fit under acquisition or sustainment.
3.22
Defence advised that in some cases, ‘acquisition and sustainment contracts were developed concurrently’. Defence also advised that as ‘the capability life cycle is a four-phased asset management process… …it is really difficult to define a hard line between where acquisition begins and sustainment begins’, and that ‘There will always be a crossover, depending on the nature of the acquisition.’
3.23
Considering which issues were considered to be acquisition or sustainment matters, Defence noted that historically two budgets would be used if the phases were running at the same time.
3.24
The Auditor-General agreed, stating:
There is always going to be, as the evidence you have just heard said, sustainment happening at the same time as procurement… …There is always going to be a bit of fuzziness around the boundary. From an accountability point of view, it is important to make that fuzziness as narrow as possible and to be clear around what has actually occurred. …
3.25
Further, the ANAO advised the Committee that it had commenced an audit on the implementation of the First Principles Review, and was due to report in the first quarter of 2018.
3.26
Defence confirmed that there was an ‘overlap grey zone’ between traditional acquisition and in-service sustainment processes, and that a total cost-of ownership approach ‘has been a very positive thing’.
3.27
The Auditor-General stated that Defence’s move to a whole-of-life costing model may improve understanding of the total cost of an asset.
I think the Department would agree that the principles on which they undertake procurement are to look at it from a whole-of-life basis… …the move of the Department to looking at procurement activities from a whole-of-life cost basis rather than simply the acquisition and the sustainment being separate.
First Principles Review
Implementation of First Principles Review
3.28
Defence provided a brief summary of the reforms arising out of the First Principles Review:
Disbanding Capability Development Group and integrating its functions into the Vice Chief of the Defence Force Group, Services, Strategic Policy & Intelligence Group, and CASG.
Disbanding the Defence Materiel Organisation and transferring its core responsibilities of capability delivery to Capability Acquisition and Sustainment Group.
The creation of Force Design Division to support the Vice Chief of the Defence Force to deliver capability aligned to strategy and resource guidance as directed by the Government.
Capability Managers being responsible for the management of capability throughout the life cycle, including capital and sustainment funding.
The appointment of the Vice Chief of the Defence Force as the Joint Capability Authority.
The establishment of the Investment Committee, chaired by the Vice Chief of the Defence Force – Membership includes all Capability Managers, Deputy Secretary CASG, and Deputy Secretaries from the Department of Prime Minister & Cabinet and the Department of Finance. Treasury is provided with all the relevant documentation prior to each monthly meeting.
Central Agency briefings, which occur both before and after each Investment Committee meeting, prior to Government consideration.
3.29
The Committee noted the expected two year timeframe for the implementation of the First Principles Review appeared to be unrealistic based on the comments of the Auditor-General. Defence replied that while it had been able to achieve all of the outcomes of some recommendations, there were some that involved longer term cultural change. Defence noted that issues that would ‘outlive the two-year implementation period’ were identified in weekly meetings with the Secretary and Chief of the Defence Force.
3.30
Vice Admiral Ray Griggs of Defence noted that the review team set the two-year implementation period ‘mainly to ensure we maintain momentum’, and stated that there were some recommendations of the First Principles Review that were always not going to be completed within that period, and that Defence would not ‘fall into the trap of just ticking them off’.
3.31
The Committee was advised that, to aid in the delivery of long-term changes expected as a result of the First Principles Review, a five year moratorium on further reviews had been declared in 2015.
3.32
The ANAO noted that Defence had engaged a US-based engineering company Bechtel to help with the implementation of the First Principles Review recommendations. Further, the ANAO reported that Defence’s contracts with Bechtel were not performance-based, with the Commercial Division in CASG expressing concern that Defence needed to clearly articulate its ‘objective/end state’ before agreeing to contracts of this cost and magnitude. Defence advised the ANAO that it would have been difficult to implement a performance-based contract in this instance, as the purpose was to develop Defence’s capacity, rather than have Bechtel deliver capacity and capability itself.
3.33
At the public hearing the Committee noted the engagement of Bechtel to provide services, and the ANAO’s finding that the contract signed to engage Bechtel was not performance-based.
3.34
Defence informed the Committee that Bechtel undertook work for the United Kingdom’s Defence Equipment and Support (DE&S) that was very similar to that required to assist with the First Principles Review and sustainment. Further:
The attraction of using Bechtel was the fact that they had done that with another large, comparable acquisition, sustainment and military organisation. …it’s not performance based… …we have seen value for money out of the work that they have done for us, and we continue on that basis.
3.35
Defence stated that Bechtel was working on acquisition and sustainment reporting reform, the smart buyer model, and ‘a range of different organisational structures of the organisation’, consisting of ‘almost all the reform activities inside the acquisition organisation’. Later in the hearing, Defence clarified that Bechtel was on a Defence procurement panel.
3.36
The Committee noted a Bechtel employee had been seconded to CASG. Defence replied that the rationale for this was because another recruitment process to find an employee to run the major projects office had not yielded a suitable candidate, that the employee from Bechtel was suitable and had been engaged for a period of time, and that it was planned for the role to be advertised within the next six months.
Realising Savings and Cultural Change
3.37
Defence advised the Committee that the Government requested that it reduce its spending on consultants and contractors by 10 per cent over the next four years, and that its consultant and contractor bill over the next four years was $673.9m.
3.38
Defence also noted that it remained focused on getting ‘the most efficient, effective service for the capability manager’ and that in some cases that would mean outsourcing activities to industry.
3.39
Further, Mr Kim Gillis of CASG stated:
What we’ve got to do is ensure there is a clear barrier between those things which are governance, which are the accountability and responsibility of Commonwealth officials, and those things which can be outsourced from a transactional perspective. We’ve got to make sure that’s clear. As a result of that, we have updated our internal guidance to staff to ensure that those barriers and delineations are kept clear.
3.40
Mr Gillis also noted that over time, some longer-term sustainment contracts did not provide value for money:
…some of the older contracts; you sign up a contract and that contract might last for five or 10 years – the industry partner might be making a better profit than they really should be, because we haven’t been driving their performance.
3.41
However, Defence noted that improvements had been made in this area through performance based contracting, and that some of the contracts delivered by Defence over industry ‘are genuinely best practice in the world because they drive exactly the right performance.’
3.42
Mr Kim Gillis of Defence also noted the importance of cultural change in realising savings:
I want people to think like a smart buyer. I want people to understand that industry is a fundamental input to capability. I want people to understand that, when we do have a failure either in acquisition or sustainment, we are reporting it openly, honestly and very early. Those are the things that we have to make sure are embedded into the culture of the organisation.
Systems Program Offices
3.43
Systems Program Offices (SPOs) manage the acquisition, sustainment and disposal of specialist military equipment through internal work and commercial contracts. In most cases, a major platform, such as an aircraft type or class of ship is managed by a single SPO. This SPO maintains a relationship with industry, which provides maintenance, spares, engineering, and other support. At the time of the ANAO audit, there were 64 SPOs.
3.44
Defence noted that transformation of SPOs wasn’t completed inside the two year period identified by the First Principles Review, as ‘it takes so much work to restructure what are sometimes $100 million contracts and to restructure the organisation around getting that efficiency.’ As a result, Defence noted that it would take at least the ‘next two to three years’ to finalise the process.
3.45
Mr Kim Gillis reported that while some of its SPOs had ‘room to improve’, some were ‘best practice’:
…some of our SPO activities and performance-based contracting not only have won international contracting awards… …but have now been identified as - our performance-based contracting is world’s best practice. Now, we’re not consistent across the board. That is the issue. We have pockets of excellence.
3.46
The Committee was advised of the process Defence had undertaken in reforming SPOs. Defence initially examines structure and staff skills/experience, determining who should undertake work, then Defence examines the contractual structures for each sustainment activity, trying to improve performance and find efficiencies.
3.47
Defence updated the Committee on progress towards SPO reform, stating that 78 SPOs were reduced to 64, and of that 64, 20 had already gone through the reform process, with 44 remaining. Mr Kim Gillis of Defence noted that a one-size-fits-all approach was not applicable in SPO reform:
There will be a layer of variation. A Systems Program Office that is looking after a single platform will have to be structured very differently to a Joint Systems Division [SPO] that is looking at Army, Navy, Air Force, communication equipment and satellite communication equipment – it might have 50 or 60 different products out there. I couldn’t make them perfectly cookie-cutter as they went through, but I’m trying to get the structures as close, and their reporting mechanisms as consistent, as I can possibly make them.
3.48
Defence also indicated its commitment to implementing a review cycle on SPOs to review the performance structure and contractual model to provide ‘efficient, effective, safe, capable sustainment response.’
Staffing
3.49
At the public hearing, Defence advised that reports had indicated that staff skill levels were ‘not where [they] should be across the Systems Program Offices’, and that while there were ‘outstanding’ staff, that some hadn’t been ‘skilled’ effectively enough to function in reformed SPOs. Accordingly, Defence was focusing on changing staff responsibilities and enhancing skill levels, moving from ‘transactional activities to actually undertaking more of a supervisory, governance role’. This approach was beginning to yield ‘a layer of expertise and centralisation of program management, logistics and governance so that we can make sure that those skills are ensured in the organisation.’ This, in turn, had led to Defence deploying staff more flexibly, enabling the Department to ‘move people with the right skills to the right job, as opposed to having them in a particular job and then stay there for 10, 15 or 20 years whether the work is there or not.’
3.50
At the public hearing, the Committee noted Defence did not include staffing expenses in SPO reporting. Defence advised that while this was historically true, Defence was requiring that all new projects include staffing costs, noting the First Principles Review had found that some SPOs were spending more on staff salaries than what was actually being purchased. Further, Defence noted that this new approach to cost staffing expenditures in SPOs included civilian, military and contracted labour.
3.51
The ANAO stated that it was ‘heartening’ to hear that Defence was moving to report on a full cost basis that which is being purchased at the decision-making stage.
Smart Sustainment and project classifications
3.52
In early 2010, Defence began its Smart Sustainment Reforms, expecting to yield $5.5 billion in savings to be reallocated within Defence over the following decade – this was the largest single source of savings under the broader Strategic Reform Program. To facilitate these reforms, Defence made $360m of ‘investment funds’ available for projects that would yield net savings.
3.53
The ANAO noted that a Defence review warned that most SPOs did not have the capacity to undertake Smart Sustainment initiatives, and that a later review by an external consultant found Defence did not track expenditure or benefits from the $360m allocated to support the program.
3.54
The ANAO found that $360m seed funding allocated to drive Smart Sustainment reforms were unaccounted for:
Notwithstanding Defence’s effort to keep track of the claimed and planned savings from Smart Sustainment, Defence has not been able to provide the ANAO with adequate evidence that the savings were realised.
3.55
At the hearing, Defence noted that $20.2bn worth of benefits had been identified from the broader program. In response to questions on notice from the Committee at the public hearing, Defence provided a breakdown of how the Smart Sustainment funds were allocated:
Of this funding, $241.7m was allocated to initiatives to deliver the savings under the program, $126.2m was returned to the Defence budget for reallocation and $1.6m was returned to the Federal budget at Mid-Year Economic and Fiscal Outlook 2011-12 as part of a one-off efficiency dividend.
The $241.7m was allocated to initiatives including:
The Air Force Improvement Program (approx. $69m)
The Navy Mine Hunter Coastal and ANZAC Capability Improvement Programs (approx. $95m)
Army initiatives such as the Repair Pool Feasibility Project (approx. $36m)
Defence Materiel Organisation’s Lean Continuous Improvement Program, Advanced Inventory Management System upgrade, the Efficient Order Quantity Review, Sustainment Performance Management System implementation and the Sustainment Complexity Review (approx. $35m).
3.56
When an acquisition project is commenced, it is classified by Defence as Commercial off the Shelf, Military off the Shelf or Developmental. The Committee queried how projects were classified, with Defence advising that it was ‘a philosophical definitional issue’, and ‘as to whether things are going to neatly fall into three little categories, it’s very hard to do this to everyone’s satisfaction’
3.57
The ANAO agreed, stating ‘it’s sometimes difficult to find an element of purity in those definitions’ and ‘one of the principle problems is that there is always… …some difficult technical developmental character, even in a commercial off-the-shelf process.’
3.58
Defence suggested that a benefit of the Smart Buyer approach was that it was possible to have more of an appreciation of ‘the fundamental inputs to capability of where the risks lie’, and that this approach may lead to Defence ‘steering away’ from developmental projects as a matter of risk.
3.59
Defence also noted that while developmental projects carried risk, that they also had potential to deliver superior capability, noting the anti-ship missile defence project on the ANZAC frigates constituted a ‘world-beating sovereign capability for Australia.’
Internal reporting
From Quarterly Performance Reports to Monthly Reporting
3.60
The audit considered Defence’s internal Quarterly Performance Report in detail, concluding that the report was:
…neither timely – it is more than 50 days old by the time it gets to Ministers – nor clear – it is dense with acronyms and jargon. Some terms are used in an unusual way, for example, expenditure of more funds than had been budgeted is referred to as an ‘overachievement’.
3.61
Mr Kim Gillis of Defence agreed with the Auditor-General’s findings in relation to the Quarterly Performance Report, noting that Defence’s IT systems were now able to perform effectively and deliver a better reporting regime:
Our IT systems are very difficult to change inside Defence because of the historical nature of them. Therefore, now we have got this online, by the end of this year we will be in a situation where we will have a much better reporting system. I welcome the auditor to have a look at our new system and giving his and his team’s views on whether we have actually made the changes.
3.62
Defence noted that there had been ‘variability’ across SPOs identified by the ANAO, and that it was difficult to obtain ‘an absolute standardised view’ of how sustainment was being handled across Defence. Defence stated that this had led to Defence moving from a monthly reporting system to an automated sustainment monitoring performance report. Data is obtained directly from the schedules of activities, and financial data is unfiltered. Defence stated that as a result: ‘the division heads get a much clearer performance metric and we can be much clearer to the capability managers.’
3.63
Further, Defence noted consistency was the ‘underlying principle’ of the new system, and that obtaining consistency removed individual interpretations of data, and ensured the system became ‘data driven’. Defence indicated that this system provided ‘timely and accurate’ information to capability managers and the government.
3.64
The Committee noted the ANAO finding that over expenditure had been characterised as “overachievement” in monthly reporting, with Mr Kim Gillis stating:
It is a significant issue that we’re trying to address… …in respect to achieving sustainment budget, there was a mindset about achieving full expenditure of sustainment budget in year without maybe necessarily focusing on the efficiency and effectiveness of that outcome.
…There have been instances where people, to achieve their budget, have spent money at the end of the financial year. That is not uncommon across all aspects of public administration. It’s just the fact that we spend so much money those issues are significant. But it is behaviour that I want to drive out of the organisation…
3.65
The ANAO stated that there were inconsistencies in Defence about which matters were included in quarterly internal management reporting.
3.66
Further, the ANAO stated:
We’ve talked about the quality of internal reporting and monitoring as the foundation stone, for example, for accurately reporting up to management within Defence and up to Ministers. We see this audit as a work in progress, for sure, but what is potentially a sound basis for doing those things.
Senator Dean Smith
Chair
1 March 2018