Chapter 3

The Defence reviews and fluid reform agenda

3.1        This chapter provides an overview of the Defence reviews as defined in the inquiry terms of reference. It details the respective reviews, their findings and recommendations whilst also providing a summary of the many other reviews that have been conducted or which are currently underway. The chapter concludes by acknowledging the fluid reform agenda and its impact on the Defence procurement process.

Background to the Kinnaird, Mortimer, Pappas and McKinsey reviews

Kinnaird Review

3.2        In December 2002, Malcolm Kinnaird, AO was commissioned to conduct a review of the 'problems associated with major Defence acquisition projects'. Kinnaird and his team submitted their Report of the Defence Procurement Review or Kinnaird Review on 15 August 2003 and the report was subsequently released on 18 September 2003. The review made ten recommendations and concluded that as there is no single cause of the failures in the development of capability and the acquisition and support of defence equipment, there is no single remedy to ensure that the problems will not recur.[1]

3.3        Kinnaird set in train a reform agenda that provided a key reference point for future reviews. The main thrust of his recommendations were directed at:

• improving the quality of information to inform decision-making, especially government;
• strengthening the capability definition and assessment function;
• developing reliable whole-of-life costs which are taken into consideration throughout a project's life;
• recognising off-the-shelf acquisitions as an integral part of procurement considerations;
• attracting project management skills to DMO;
• improving the accountability of Capability Managers; and
• transforming DMO into a performance driven organisation.  

3.4        The Kinnaird Review recommendations led to a number of reforms including the two-pass approval system and creation of the Capability Development Group (CDG) headed by a three-star officer or equivalent. Lieutenant General David Hurley was appointed Chief of the CDG in December 2003 whilst the Defence Procurement Advisory Board was established in March 2004. Kinnaird's recommendations also led to the creation of the Defence Materiel Organisation (DMO) as a prescribed agency under the Financial Management and Accountability Act 1997.[2] The Minister for Defence, the Hon. Stephen Smith MP (the minister) stated in May 2011 that together with these initiatives, most of the Kinnaird reforms have been implemented and have had a 'positive impact'.[3]

Mortimer Review

3.5        In May 2008, the government commissioned David Mortimer, AO to conduct a formal evaluation into the effectiveness of ongoing reforms in the DMO implemented following the 2003 Kinnaird review. Mortimer and his team were also asked to provide advice on identifying further potential reforms to the acquisition and through-life support of defence equipment. The Defence Procurement and Sustainment Review or Mortimer Review was tabled in Parliament on 23 September 2008. The 46 recommendations of the Mortimer Review were directed at making the DMO 'more business-like and imposing commercial discipline on the defence procurement and sustainment processes'.[4]

3.6         Both the Kinnaird and Mortimer reviews considered Defence procurement through the ADF capability life cycle from the initial stages of strategic assessment and where a need is identified to address a current or potential capability gap. Both reviews make a series of findings and recommendations directed at strengthening the respective phases as well as the overall life cycle. For the committee's purposes, it will focus on three main phases of the capability life cycle—strategic analysis, needs and requirements; acquisition; and sustainment.[5]

3.7        The Mortimer Review recognised that the implementation of Kinnaird's recommendations resulted in wide-ranging reform and improvement in the capability development process in the Defence and the acquisition process in DMO.[6] However, Mortimer refuted claims that the two-pass process implemented following the Kinnaird Review had solved all the problems. In evidence to support this position, he drew on two examples including the acquisition of tactical unmanned aerial vehicles (Project JP 129) where he found continued problems of scope creep and poor commercial practice leading to the termination of the contract. In the second example concerning Phase 3 of Project Land 121, Mortimer found that scope creep and poor capability definition and commercial practice resulted in the need to refresh the tender.[7]

3.8        Mortimer identified five principle areas of concern ranging from:

... inadequate project management resources in the Capability Development Group, the inefficiency of the process leading to government approvals for new projects, shortages in DMO personnel, to delays due to inadequate industry capacity and difficulties in the introduction of equipment into full service.[8]

3.9        Of the 46 Mortimer recommendations, the government agreed in full to 42 and in part to three. On 6 May 2011, the minister stated that whilst many of the recommendations had been implemented, some had yet to be fully implemented and that Defence would accelerate the implementation of all outstanding agreed Mortimer recommendations including:

• Project directives issued by the Secretary of Defence and Chief of the Defence Force (CDF) to ensure Defence acquisitions progress according to government direction;
• Benchmarking all acquisition proposals against off-the-shelf options where available.[9]

Pappas Report

3.10      On 30 July 2008, Mr George Pappas was appointed to lead an independent audit of the Defence budget. Known as the Pappas Report, the 2008 Audit of the Defence Budget report was delivered to the government in April 2009. The objective of the audit was to advise ministers on the efficiency and effectiveness of and future risks associated with the Defence budget and to make recommendations to improve arrangements for managing the Defence budget.[10] The audit contains over 120 recommendations of which the majority are to be delivered through the Strategic Reform Program (SRP discussed below).[11] Mr Pappas serves as the chair of the Defence Strategic Reform Advisory Board which is responsible for overseeing the SRP.[12]

3.11      The McKinsey Report (2010) is a benchmark study undertaken in 2008 and 2009 comparing the performance and productivity of defence ministries worldwide. The report ranked Australia near the bottom of an international league table of 33 countries on defence performance and equipment acquisition, finding Australia equal last with the US on defence spending.

3.12      According to then Defence Secretary, Dr Ian Watt, McKinsey published the report without consulting Defence which he noted was 'at complete odds with the advice that McKinsey's had given us in relation to the defence budget audit' or Pappas Report which was conducted by McKinsey Australia. According to Dr Watt, the Defence budget audit had considered a number of benchmarks which would have put Australia in the realm of 'world best practice'. Further, McKinsey International had published the report without the benefit of input from McKinsey's Australian practice. McKinsey International subsequently sent a letter of apology to Dr Watt and then CEO of DMO, Dr Stephen Gumley.[13]

3.13      McKinsey subsequently republished the article with Australia withdrawn from the list and the statement 'Australia has been removed because the data is incorrect for Australia and the methodology does not apply to Australia'.[14]

Implementing and monitoring Defence reviews recommendations 

3.14      Defence informed the committee that the majority of the recommendations of the respective reviews (Kinnaird, Mortimer, Pappas) have been agreed to and implemented. Given the large number of procurement projects underway, Defence observed, however, that the effects of the respective reviews 'which primarily affect new projects, take some time to impact on the procurement system as a whole'.[15]

3.15      In terms of the mechanism to monitor the implementation of the reforms and recommendations of the respective reviews, Defence held that it has established the following bodies:

• In relation to the Kinnaird Review, the creation of a Defence Procurement Advisory Board to support the establishment of the DMO and report to the Ministers for Defence and Finance and Deregulation at regular intervals on the implementation of the Kinnaird Review recommendations.[16]
• In relation to the Mortimer Review, the Strategic Reform Program (SRP) is delivering reform initiatives through a number of 'streams':

  The SRP incorporates the Mortimer Review reforms as one of the streams. SRP reforms target all stages of the capability life cycle and are designed to enhance alignment between strategic planning and capability development. The SRP also contains recommendations to improve the procurement process and increase the effectiveness and efficiency in the maintenance of defence capability.  Reform streams that will contribute to these objectives are the Strategic Planning, Capability Development, Mortimer, and Smart Sustainment streams.[17]

• In relation to the Pappas Review, the recommendations are being delivered through the SRP.[18]

Strategic Reform Program

3.16      The SRP was announced in 2009 as a ten-year plan to deliver gross savings of approximately $20 billion. As part of the SRP, reforms and initiatives in the areas of accountability, planning and productivity, directed at improving management and at greater efficiency and effectiveness, were designed to deliver such saving that would in turn:

...be reinvested to deliver stronger military capabilities, to remediate areas where there has not been enough funding in the past and to modernise the Defence enterprise 'backbone', all of which are essential to support the fighting force.[19]

3.17      The SRP will be delivered through 15 reform streams each to implement a program of reform. As previously noted, implementation of the Mortimer Review recommendations is one of the SRP streams. Defence noted that as of 18 May 2011, 32 of the process Mortimer recommendations had been fully implemented with an additional two transferred to the SRP Stream whilst 11 recommendations remain 'on track' for implementation in 2012.[20]

3.18      According to Defence, some streams of the SRP will deliver direct savings that have been earmarked for reinvestment in Force 2030 whilst others will put downward pressure on costs through improved governance, planning and processes. Streams that drive more efficient and effective outcomes but do not have cost reductions attached to them include Strategic Planning; Capability Development; Procurement and Sustainment (Mortimer); Preparedness, Personnel and Operating Costs; Intelligence; Science and Technology; Estate; and Output focused budget model.

3.19      Streams that drive more efficient and effective outcomes and have cost reductions directly attached to them include Smart Sustainment; Non-Equipment Procurement; Workforce and Shared Services; Information and Communications Technology; Reserves; Logistics; Defence Savings Program.[21]

3.20      According to Defence, improved processes and activities continue to be developed under the SRP reform streams, the Mortimer (Procurement and Sustainment) Reform Stream and the Capability Development Reform Stream. These reforms have been captured in the promulgation of an updated Defence Capability Development Handbook (DCDH) to 'record the improved processes and governance arrangements and provide guidance on capability development documentation'.[22]

3.21      As part of a comprehensive second report, the committee intends to consider what Defence means by 'improved governance' and 'governance arrangements' in the context of the SRP.

Defence reforms announced in 2011

3.22      A number of additional reforms were announced in 2011 and include reforms to:

• project management accountability (announced on 6 May);
• strengthen the Projects of Concern process (announced 29 June);
• the disposal of military equipment (announced 29 June);
• strengthen Australian industry (announced 29 June);
• support ship repair and management practices—implementing recommendations from the Plan to Reform Support Ship Repair and Management Practices or Rizzo Report (announced 18 July);
• improve accountability in Defence—implementing the recommendations of the Review of the Defence Accountability Framework or Black Review (announced 9 August).

3.23      The 'project management accountability reforms' which are also called the 'Accountability and Procurement' reforms by the Minister for Defence,[23] entail implementation of outstanding Kinnaird and Mortimer recommendations including that of benchmarking proposals against off-the-shelf options. However, they also include additional reforms set to build on the Kinnaird and Mortimer recommendations. Focused on improving project management and minimising risk at the start of a project whilst identifying problems early, they include the:

• introduction of a two-pass approval system for minor capital projects valued between $8 million and $20 million;
• implementation of an Early Indicators and Warning system;
• expansion of the Gate Review system; and
• introduction of Quarterly Accountability Reports.[24]

3.24      On 19 July 2011, the Minister for Defence detailed the government's reform agenda for Defence and the initiatives which are either linked to or in addition to the ongoing SRP program. The reforms, additional to those mentioned above, include:

• Procurement and Sustainment—focused on reforms to the Defence budgeting process, capability acquisition and development, and to the maintenance and sustainment of equipment in service;
• The Defence Budget—improving and reforming Defence's planning and budgeting processes;
• Defence Capability Plan (DCP)—including efforts to reduce the level of over-programming in the DCP;
• Linking the DCP to Defence Planning Guidance—linking updates of the DCP to the Defence Planning Guidance to ensure that information provided to industry is based on the latest national security tasks;
• Force Posture Review—to examine strategic and security considerations and assess the impact of the ADF's Force Posture of these considerations and make recommendations in relation to the basing options across Australia.[25] 

3.25      The reviews yet to be completed and the reforms that flow from them should also be noted. These include the government's response to the Coles Review of the Sustainment of Australia's Collins Class Submarines and the Shared Services Review.[26]

Other relevant reviews and analysis

3.26      There are a number of other reviews of Defence which have addressed the issue of Defence procurement which are not only pertinent to this inquiry but also produce recommendations for Defence reform.

3.27      As part of its response to the Mortimer Review, the government stated that the Australian National Audit Office (ANAO) would be invited to audit the progress of reform at nine and 18 months post-commencement and report its findings against the agreed plan.[27] Further, the DMO and ANAO Major Projects Report (MPR) which is published annually, provides a performance overview of selected major defence capital acquisition projects managed by the DMO. The Parliamentary Joint Committee on Public Accounts and Audit (JCPAA) is charged with reviewing the MPR, making findings and recommendations for improvement. In addition, ANAO audit reports often include recommendations to Defence. The most recent Performance Audit Report 57 titled Acceptance into Service for Navy Capability serves as an invaluable resource on matters of Defence procurement and makes eight recommendations designed to improve Defence's management of the acquisition and transition into service of Navy capability.[28]

3.28      Other reviews conducted outside the Defence establishment include the committee's own 2003 Report on the inquiry into materiel acquisition and management of Defence.  The JCPAA and the Joint Committee on Foreign Affairs, Defence and Trade have both produced reports of relevance to this inquiry with recommendations to government generally and Defence specifically in relation to improving processes.

3.29      Finally, the analyses of Defence procurement matters provided by the Australian Strategic Policy Institute (ASPI) are pertinent to this inquiry. Defence has in the past commissioned ASPI to provide advice on matters such as the Public Defence Capability Plan and to make recommendations for improvements and reforms. These recommendations are also a matter for implementation by Defence.

Stocktake of reforms

3.30      In 2003, Kinnaird argued that further fundamental reform was needed to ensure that the ADF receives the capabilities it expects according to the schedule required by the government. Five years later, Mortimer concluded that reform in acquisition and sustainment should continue in order to extract maximum benefit across the capability systems life cycle. In 2009, Pappas found the need for 'fundamental reform'.[29] Recently Rizzo and Black added to the reform program. Indeed Rizzo found, among other things, that Navy had 'poor whole-of-life asset management, organisational complexity and blurred accountabilities, inadequate risk management, poor compliance and assurance, and a "hollowed-out" Navy engineering function.'[30] Black pointed to poor outcomes in Defence including delivery failures for capability projects and poor or inappropriate procurement decision-making.[31] 

3.31      As part of its stocktake, the committee wrote to the Department of Defence (department) on 6 July 2011 requesting a schedule showing the progress made on implementing the recommendations made in recent defence reviews with particular focus on the Kinnaird, Mortimer and Pappas reviews. The department responded on 21 July, but the committee was disappointed with the information provided and again wrote to the department and DMO on 22 August 2011.

3.32      In its letter, the committee informed the department that, while helpful, the information did 'not always convey a clear picture of what was being achieved'. It noted that the department's response provided no information in relation to the relevant Pappas recommendations. Further, that while the term 'fully implemented' was appropriate to describe some of the Mortimer recommendations, for others the term simply raised more questions. The committee explained that it would prefer a more informative response from Defence suggesting that a brief comment be added to each recommendation 'to put beyond doubt what is meant by fully implemented'.

3.33      On 4 October as part of a Defence Organisation response, the department provided the committee with an updated version of progress on the implementation of its reform program (see appendix 4). The committee notes, however, that with a number of recommendations made in the Mortimer review, Defence has indicated that implementation was complete. Yet, in the committee's view, some of the recommendations would form part of a continuous improvement process requiring constant attention, particularly those relating to developing the skills base. The committee is not convinced that Defence's account of the progress made in implementing its reform program has the coherence and foresight need to achieve lasting success. As an accountability measure, the report provided to the committee on the implementation of the reforms failed.

3.34      Furthermore, Defence informed the committee that in light of the Black Review's findings of challenges and weakness in Defence's end-to-end management of capability, 'a comprehensive end-to-end business process review of Defence's capability management will be undertaken'.[32] The committee has no insight into what this review is supposed to achieve and unfortunately again the focus appears to be on process.

Concerns regarding reform agenda

3.35      Many submitters maintained that the reform agenda has improved the capability development and acquisition process. Indeed, the findings and recommendations of such reviews and reports have led to what ANAO describes as 'rapid organisational change to procurement process, practice and organisational structures'.[33] These include the strengthening of existing processes, introduction of new processes, adjustment of high level command and oversight, the establishment of new committees and review boards, and the rearrangement of organisational charges. However, there remain two key concerns regarding the direction of the Defence reform agenda. First, that the reforms are focused on process at the expense of fundamentals and second, that constant reform has created fluidity rather than effect.

Focus on process rather than fundamentals

3.36      One of the major concerns raised in relation to the implementation of the reform agenda is that it has resulted in the addition of more process to an already process-bound organisation. Air Marshal Binskin noted that whilst each one of the respective reviews has increased the 'transactional costs; it has added to the process, not necessarily streamlined' it.[34] Air Commodore (Retired) Bushell noted congenital problems in Defence included an inability to manage complex projects and particularly those with any degree of system development or integration as well as difficulties in providing in-service support on time. In his view, they stem:

...directly from an entrenched, process-driven, contract centric approach to project management, rather than employing sound Project, Systems and Equipment Engineering management systems and procedures developed especially for controlling technology projects. The situation that has persisted for more than a decade is an inevitable consequence of the 'not thought through' de-skilling and downsizing of the Services and the structural changes imposed by the Defence Reform Program (DRP) and Commercial Support Program (CSP).[35]

3.37      The concern raised by submitters is that Defence has focused its efforts on implementing the reforms in relation to procurement processes without addressing the 'fundamental Defence, DMO and Service organisational structures, accountabilities and resource distribution'.[36] In this regard, Air Commodore (Retired) Bushell stated:

No study has ever been made of the proper allocation of responsibility, accountability, and the division of resources between the Services and the Defence bureaucracy where the major problems have arisen and been left to fester for over two decades.[37]

3.38      Other submissions to the inquiry noted this focus on administrative process rather than the management of outcomes and employment of necessary feedback loops. As previously noted, such feedback would otherwise provide for an accurate understanding of the status of a project and enable the effective implementation of Defence reforms.

3.39      Some submitters raised concerns that the SRP is targeting process rather than addressing the critical issue of governance. As highlighted by a number of witnesses, the existing process is considered adequate but it is not applied consistently. The question for the committee is why the gap between procedure and practice persists. Evidence suggests that it relates to poor governance and a lack of internal checks and balances upheld by suitably qualified competent and independent personnel. In its next report, the committee intends to explore this evidence.

3.40      Other unintended consequences of the reforms which ignore the fundamentals go to the heart of the dysfunction within Defence. These include the low engineering skills base brought about by outsourcing as a consequence of the Commercial Support Program introduced in 1991 and the convoluted process and lack of clarity regarding responsibility which are unintended consequences emanating from the shared service reforms of 2009. This position is articulated by Air Commodore (Retired) Bushell who stated that:

The problems being encountered have been institutionalised firstly by the fundamental models used in the management and governance of the acquisition bureaucracy, and secondly by the practice of replacing technologically skilled engineering professionals with technologically unskilled generalists. That is, the imposition of administrative process over project and systems engineering management. For more than a decade, the approaches adopted have been shown not to work, and can not be made to work.[38]

3.41      In this regard, Dr David Robinson of Engineers Australia noted that 'if we have wrong decisions made at the beginning, inappropriate technical decisions, the best management may well deliver a lemon, but that is not what you want'.[39]

3.42      The committee is concerned that implementation measures that simply look at a further change to process will not, of themselves, be successful. The committee underlines its firm view that the focus on improving process will not solve deeper, fundamental problems. The critical question is why no one is probing these underlying issues.

Effectiveness of reform agenda

3.43      Air Marshal Binskin observed that one of the problems that arises is that 'we are changing every five years but the projects take eight years to deliver' and that:

We are inside our own decision loop sometimes with the reviews rather than letting them mature for a little bit and refine the processes that you need to be able to run that system.[40]

3.44      The continual cycle of reviews and reports into Defence from which findings and recommendations are acted upon has led to an ongoing and fluid reform process. Given the frequency of such changes and average lifespan of major acquisition projects, ANAO noted that 'several changes to organisational structures and processes can occur over the life-cycle of a single project making it difficult to assess the effectiveness of any single change'.[41] The outstanding question for the committee, therefore, is what is the underlying factor that has prevented these accumulated changes from being effective? Witnesses to the inquiry indicated that they thought it was the extensive committee structure and interfaces between the many 'groups' within Defence that have an input (or indeed control) elements of the FIC that by definition are part of capability. It should be noted in this regard that the efficacy of governance over the procurement process cannot be divorced from the efficacy of governance across Defence because of the FIC interfaces.

3.45      The ANAO also recognised that managing projects in an 'environment of successive, significant organisational and management reforms can add to the complexity of the task'. It noted that given the long lifespan of most Defence acquisition projects, the full benefit of performance improvements expected from a reform are 'only likely to be realised in projects that are started following the introduction of the reform and arguably, only once the reform has been fully implemented and consolidated within Defence's practices'.[42] Other submitters to the inquiry raised concerns about the unintended consequences of the series of reforms to which Defence has been subject which have created a context of fluidity and appearance of constant change. The result is a procurement process that has been subjected to and continues to undergo numerous changes to the point where mapping the process at any given time or providing a 'snapshot' of how the process works is a task made almost impossible.

3.46      These are critical matters that the committee intends to focus on in its second and substantive report.

Defence Capability Development Handbook

3.47      The 2011 Defence Capability Development Handbook (DCDH) builds on the 2006 Defence Capability Development Manual (DCDM) to which much of the evidence before the committee refers. According to Defence, the DCDH takes into account the recommendations of the Strategic Reform Program, Mortimer Review, and the 2009 ANAO Audit of The Planning and Approval of Major Capital Equipment Projects. Defence stated the following in relation to the DCDH:

The DCDH provides guidance, and is the template of the process for the conduct of capability development in Defence. It is not in itself a policy document.[43]

3.48      Four interim versions of the DCDH were produced between February 2010 and March 2011 before the current August 2011 version was published. It remains, however, unavailable to the public. The DCDH was released before the Rizzo and Black reviews and does not, therefore, take into account the recommendations, reforms and developments emanating from those reviews.[44] Air Marshal Harvey noted that this was one of the challenges faced by Defence as there are 'always reviews and reforms going on' and that Defence would consider whether an update to the DCDH is required at the appropriate time.[45]

Committee view

3.49      The committee is concerned that when implementing the ongoing and seemingly endless reform agenda, Defence's focus has produced layers of additional administrative process without fixing deeper problems. It also means that Defence is caught in a reform roundabout where before one set of reforms can be implemented, another one takes over.

3.50      The question for the committee is how to redirect attention and energies towards addressing of the fundamentals in order to affect real change and stop the endless cycle of reviews and recommendations which have become a symptom rather than a solution to the problems before Defence.

Navigation: Previous Page | Contents | Next Page