Part II


Part II

Risk and defence procurement

Defence organisations face particular and significant challenges in managing the procurement and sustainment of their major capital equipment. For decades they have been seeking ways through risk management to improve their performance in particular to contain costs, keep to schedule and achieve a technological advantage.

The Australian defence organisation confronts the same problems as those worldwide. Its major capital equipment acquisition projects are expensive, large and complex, span many years and strive to be at the forefront of technology. While Defence recognises these difficulties, it accepts that to provide leading edge capability, it must accept a high level of procurement risk. Too often this has meant increased process and decreased decision making by informed individuals who have been advised by subject matter experts. This strategy has failed to ensure the availability and effective employment of suitably qualified/experienced people and a system to ensure that their views are heard.

Many witnesses, including the Department of Finance and Deregulation, agree that risk is inherent to defence procurement but that a key consideration is to balance the need to meet unique or specific capability requirements against the likely increase in project risk. Dr Brabin-Smith also noted that the key for Defence is to be able to judge the best balance between strategic and technical risk. He acknowledged that Defence must accept and manage this risk by having a robust risk management strategy to undergird its acceptance of risk.

In Part II of the report, the committee examines the implementation of Defence's risk management strategies against recognised best practice. It compares Defence's stated policy on risk management and the advice or direction contained in its relevant guides on procurement with practice and actions. The committee's purpose in drawing these connections is to better locate the source of Defence's acquisition problems.

Chapter 5

Risk management and good governance

5.1        For many years, risk management has been recognised as an integral part of good governance and central to an organisation's management processes. Its aim is to improve organisational efficiency and effectiveness and to limit the potential for surprises.[1] Risk management involves the actions taken to ensure 'an organisation is conscious of the risks it faces, makes informed decisions in managing these risks, and identifies and harnesses potential opportunities'.[2] It is especially important for an organisation such as Defence whose acquisition program already faces external forces that create complexity and uncertainty. Indeed, Mr Derek Woolner observed that any engineering or construction project is 'about managing risk, whether it be in Defence or in private companies'.[3]

5.2        In this chapter, the committee examines risk management in Defence organisations and its role in improving performance in procurement, especially decision-making. The committee considers Defence's policy on, and the principles that underpin, its risk management.

Risk management in defence organisations overseas

5.3        Because the acquisition of major defence assets is a high risk activity, defence organisations recognise that sound management practices can reduce the potential for poor results. Countries, including the US, Canada and the United Kingdom (UK) have recently implemented reform programs to improve their defence procurement performances. Notably, they have singled out risk management as one of the areas needing greater attention. According to the US Government Accountability Office (GAO):

...it is only through the thoughtful management of risks throughout all phases of the acquisition process that successful outcomes can be achieved.[4]

5.4        The US Department of Defense (DoD) has put in place policies and practices designed to mitigate the key risks associated with acquisition. It wants to do so by ensuring a 'more rigorous assessment of alternatives, competitive prototyping, more frequent and effective program reviews, the prevention of requirements creep, independent assessment of “technology readiness,” and better methods of testing and evaluation'.[5] For example, the US Quadrennial Defense Review Report stated:

To reduce technical risk, we will conduct a comprehensive design review, including independent reviews, to certify that the technologies involved are sufficiently mature before any program can progress to the costly final phase—engineering and manufacturing development.[6]

5.5        In 2010, the Auditor-General of Canada highlighted the need to recognise that the acquisition of complex equipment 'brings with it unique risks and challenges that need to be properly identified and managed using an appropriate procurement strategy'.[7] As one of the solutions to Canada's defence procurement problems, the Standing Committee on National Defence recommended that procurement strategies must not only identify risk, but also adopt strategies that inherently minimize risk. The government agreed with the recommendation. While suggesting that it would continue to improve its risk management, the government indicated that it was implementing a new policy on the management of projects including the requirement to consider project risk and the capacity to manage it.

5.6        The UK Ministry of Defence (MoD) recently acknowledged that to improve its overall capability it would, among other things, 'explore how to make further improvements to its project and programme management, including risk management'.

Risk management—best practice

5.7        Risk management policies, practices and tools continue to evolve and, over the years, international and country specific standards have established guiding principles to achieve best practice in this area. While the literature on this subject is extensive,[8] it demonstrates a broad consensus regarding the main steps and activities of a sound and effective generic risk management process.[9] For example, the Commonwealth Procurement Guidelines indicate that risk should be 'built into an agency's procurement processes'.[10] It states further that risk management involves 'the systematic identification, analysis, treatment and allocation of risk'.[11]

5.8        Because of the wealth of literature on risk management and the general agreement on the fundamental principles that underpin an effective risk management regime, the committee saw no need to give detailed consideration to best practice in this area. It should be noted, however, that many witnesses highlighted the need for Defence to give close attention to identifying and mitigating risk during the early stages of the procurement process, especially during capability definition. In their experience, the consequence of any failure at this stage of the procurement has the potential to surface later in the acquisition process and to cause serious disruption to a project.[12]

5.9        Based on international and Australian literature, and with a particular focus on defence procurement, the committee notes that to be effective, risk management should or must be:

  • considered from the outset or formative stage of a project when critical decisions are made that have significant implications for the overall success of an acquisition and its through-life support;[13]
  • an iterative process throughout the acquisition and sustainment of capital equipment involving the identification, analysis, mitigation planning, mitigation implementation and tracking and reporting of risk—consulting and communicating with all stakeholders on risk and risk management is important;[14]
  • comprehensive, systematic and applied consistently across the entire organisation at the enterprise, business and operational level;
  • broad-based ensuring that all the various factors associated with a defence procurement are assessed for risk—'even those considered as obvious need to be identified and treated'—budget, schedule, technical requirements, workforce, environmental, infrastructure, contract and stakeholder relations;
  • fully integrated and embedded in an organisation's culture so that risk management policy and practice is part of management thinking and actions and permeates all levels of the organisation— enterprise level, function level or business unit level—senior managers in particular must show leadership and commitment and managers at all levels must take responsibility;[15] and
  • part of a continuous improvement system where experiences in risk inform revised risk assessment and management strategies—this means that lessons must be learnt from previous experience and applied to future decisions and actions regarding risk management.[16]

Committee view

5.10      Clearly, risk management is a part of good governance and not an add-on. Although the acquisition of major defence assets is a high risk activity, sound management practices can reduce the potential for poor results. Thus, responsible for large and complex projects involving cutting edge technology, defence organisations have a very real interest in managing risks. Failure to do so can result in poor project performance—cost overruns, schedule slippage or shortfalls in capability. Thus, it is essential for an organisation to be well placed to anticipate, understand and manage risk. To do so effectively, it should have a sound risk management framework that binds all forms of procurement undertaken by the organisation and be front and centre of decisions for managing its projects effectively.[17] And as pointed out elsewhere, good risk management in the defence environment will occasionally need to tolerate some failure. For example, the airborne warning and control system where, despite the risk and some failure, a lot of the capability sought was eventually achieved—though perhaps it could have been better assessed at the outset.

Risk management in Defence

5.11      In Australia, the Defence Procurement Policy Manual defines risk in the defence context as being concerned with the 'things that can go wrong' to its projects and which may prevent the project from being a success. It states that the government considers that a successful project is one that 'delivers a fit-for-purpose capability, as approved by Government, within the approved budget and schedule'.[18]

Policy

5.12      In 2009, Standards Australia published a revised version of its principles and guidelines on risk management.[19] The Commonwealth Procurement Guidelines and companion guides such as the Commonwealth Policy Framework for National Public Private Partnership also advocate the use of risk management and provide advice on its application. Consistent with these guidelines, Defence has produced a number of key documents that further underscore the importance of understanding risk and its effective management. They include:

  • Defence Procurement Policy Manual;
  • Defence Capability Development Handbook;
  • Technical Risk Assessment Handbook;
  • DMO Project Management Manual; and
  • DMO instructions.

5.13      Based on these documents, Defence has certainly demonstrated that it is an organisation that recognises the importance of risk management as an indispensible part of effective governance that underpins sound decisions.[20] For example, the Defence Procurement Policy Manual (the Manual) clearly states Defence's commitment to 'a comprehensive, coordinated and systematic approach to risk management'.[21] It recognises that sound risk management is a vital component of good corporate governance and that a 'well developed and managed risk management plan will lead to informed decision-making to ensure the desired result is achieved'.[22]

5.14      Air Marshal Harvey acknowledged that Defence must manage risk. When it comes to the practical application of Defence's policy on risk, he explained that Defence has 'a very structured approach' that has been refined in line with previous reviews and Defence's internal work. He also noted that Defence's consideration of risk is broad, which covers cost, schedule, capability, technical, workforce and overall programmatic risk.[23]

Guidelines

5.15      The Defence Capability Development Handbook (the handbook) sets out the specific steps to be taken with regard to risk management. This document is a guide to the capability development body of knowledge, best practice and processes for Defence. It provides directions and offers advice on risk in defence procurement. According to the handbook:

Command and management processes at all levels are required to plan, apply, measure, monitor and evaluate the functions an agency performs, with due cognisance of risk assessment and subsequent risk management.[24] 

5.16      It states that 'every proposal must ensure that Government is aware of the risk it accepts in making an investment decision. Risks must be measured, mitigated and managed to ensure there is a tolerable risk-return balance'.[25] 

5.17      Unlike the procurement policy manual, the handbook does not have a discreet section on risk management. It deals with the implementation or practical application of risk management at every phase of the acquisition process. It recognises the need to consider risk early and for it then to be a logical and sequential process throughout the capability development cycle.

5.18      The committee examined Defence's Procurement Handbook and related documents—DMO Project Management Manual and Project Risk Management Manual. It looked at risk management from the needs phase through to entry to the DCP, first and second pass approval, acquisition, tendering and contracting to delivery, including the use of early warning systems designed to stop projects becoming projects of concern. It considered the various panels and committees that review the project proposals at milestones during the acquisition process including the Options Review Committee (now replaced by the Project Initiation and Review Board), the Capability Gate Review Boards, the Defence Capability Committee (DCC), the Defence Capability and Investment Committee, Service Chiefs and Group Heads and finally the Secretary of Defence and CDF who clear a submission for government consideration and final approval.

5.19      It is clear that, although the committee has not described step by step Defence's risk management process as set down in its manuals and guidelines, it found that the contents of the documents align with good practice. For example, the committee notes the comprehensive coverage Defence gives to risk management in its policy and practice guidelines. Defence clearly recognises risk management as:

  • integral to efficiency and effectiveness; and
  • a means that enables agencies 'to proactively identify, evaluate and manage risk, opportunities and issues arising out of procurement related activities'.[26]

5.20      Consistent with the key principles of sound risk management, the committee found that the handbook, DMO Project Management Manual and DMO Instructions recognise the importance of:

  • considering risk from the earliest stages of procurement planning;
  • monitoring risk and its treatment on a systematic basis throughout the procurement process—risk management is treated as a continuing process, with opportunities to re-evaluate risks at key stages of the procurement process;
  • taking account of all aspects of risk including costs, schedule, capability, programmatic and workforce; and
  • providing senior leaders and the government with sufficient and reliable information upon which to make decisions.

5.21      There can be no doubt that Defence's stated policy on risk management and the guidelines and handbooks intended to assist officers implement the policy is consistent with international and Australian standards.

Conclusion

5.22      In light of Defence's risk management policy, the practical guidance provided in its procurement handbook and the in-built review structures; it would appear reasonable to assume that risk management is a prominent and essential element of Defence's procurement culture. If implemented properly, Defence's policy, supporting documents and practices should work effectively to mitigate risks. Evidence, however, suggests otherwise. Indeed, the poor performance of some major projects, detailed in chapter 2, indicates that risk management may not have been as robust as it should have been—for example, cautionary advice from domain experts not understood, downplayed, misplaced or ignored as it moves up the decision-making hierarchy. On countless occasions, the ANAO has noted that this repeated failure to identify or acknowledge risk is simply a manifestation of bad management in an unaccountable system.

5.23      In the following chapter, the committee begins its examination of the underlying causes of poor performance.

Navigation: Previous Page | Contents | Next Page