Part II
Risk and defence procurement
Defence organisations face particular and significant
challenges in managing the procurement and sustainment of their major capital
equipment. For decades they have been seeking ways through risk management to
improve their performance in particular to contain costs, keep to schedule and
achieve a technological advantage.
The Australian defence organisation confronts the same problems
as those worldwide. Its major capital equipment acquisition projects are
expensive, large and complex, span many years and strive to be at the forefront
of technology. While Defence recognises these difficulties, it accepts that to
provide leading edge capability, it must accept a high level of procurement risk.
Too often this has meant increased process and decreased decision making by informed
individuals who have been advised by subject matter experts. This strategy has
failed to ensure the availability and effective employment of suitably
qualified/experienced people and a system to ensure that their views are heard.
Many witnesses, including the Department of Finance and
Deregulation, agree that risk is inherent to defence procurement but
that a key consideration is to balance the need to meet unique or specific
capability requirements against the likely increase in project risk. Dr
Brabin-Smith also noted that the key for Defence is to be able to judge the
best balance between strategic and technical risk. He acknowledged that
Defence must accept and manage this risk by having a robust risk management
strategy to undergird its acceptance of risk.
In Part II of the report, the committee examines the
implementation of Defence's risk management strategies against recognised best
practice. It compares Defence's stated policy on risk management and the advice
or direction contained in its relevant guides on procurement with practice and
actions. The committee's purpose in drawing these connections is to better
locate the source of Defence's acquisition problems.
Chapter 5
Risk management and good governance
5.1
For many years, risk management has been recognised as an integral part
of good governance and central to an organisation's management processes. Its
aim is to improve organisational efficiency and effectiveness and to limit the
potential for surprises.[1]
Risk management involves the actions taken to ensure 'an organisation is
conscious of the risks it faces, makes informed decisions in managing these
risks, and identifies and harnesses potential opportunities'.[2]
It is especially important for an organisation such as Defence whose
acquisition program already faces external forces that create complexity and
uncertainty. Indeed, Mr Derek Woolner observed that any engineering or
construction project is 'about managing risk, whether it be in Defence or in
private companies'.[3]
5.2
In this chapter, the committee examines risk management in Defence organisations
and its role in improving performance in procurement, especially
decision-making. The committee considers Defence's policy on, and the
principles that underpin, its risk management.
Risk management in defence organisations overseas
5.3
Because the acquisition of major defence assets is a high risk activity,
defence organisations recognise that sound management practices can reduce the
potential for poor results. Countries, including the US, Canada and the United
Kingdom (UK) have recently implemented reform programs to improve their defence
procurement performances. Notably, they have singled out risk management as one
of the areas needing greater attention. According to the US Government
Accountability Office (GAO):
...it is only through the thoughtful management of risks
throughout all phases of the acquisition process that successful outcomes can
be achieved.[4]
5.4
The US Department of Defense (DoD) has put in place policies and
practices designed to mitigate the key risks associated with acquisition. It
wants to do so by ensuring a 'more rigorous assessment of alternatives,
competitive prototyping, more frequent and effective program reviews, the
prevention of requirements creep, independent assessment of “technology
readiness,” and better methods of testing and evaluation'.[5]
For example, the US Quadrennial Defense Review Report stated:
To reduce technical risk, we will conduct a comprehensive
design review, including independent reviews, to certify that the technologies
involved are sufficiently mature before any program can progress to the costly
final phase—engineering and manufacturing development.[6]
5.5
In 2010, the Auditor-General of Canada highlighted the need to recognise
that the acquisition of complex equipment 'brings with it unique risks and
challenges that need to be properly identified and managed using an appropriate
procurement strategy'.[7]
As one of the solutions to Canada's defence procurement problems, the Standing
Committee on National Defence recommended that procurement strategies must not
only identify risk, but also adopt strategies that inherently minimize risk.
The government agreed with the recommendation. While suggesting that it would
continue to improve its risk management, the government indicated that it was
implementing a new policy on the management of projects including the
requirement to consider project risk and the capacity to manage it.
5.6
The UK Ministry of Defence (MoD) recently acknowledged that to improve
its overall capability it would, among other things, 'explore how to make
further improvements to its project and programme management, including risk
management'.
Risk management—best practice
5.7
Risk management policies, practices and tools continue to evolve and,
over the years, international and country specific standards have established
guiding principles to achieve best practice in this area. While the literature
on this subject is extensive,[8]
it demonstrates a broad consensus regarding the main steps and activities of a
sound and effective generic risk management process.[9]
For example, the Commonwealth Procurement Guidelines indicate that risk
should be 'built into an agency's procurement processes'.[10]
It states further that risk management involves 'the systematic identification,
analysis, treatment and allocation of risk'.[11]
5.8
Because of the wealth of literature on risk management and the general
agreement on the fundamental principles that underpin an effective risk
management regime, the committee saw no need to give detailed consideration to
best practice in this area. It should be noted, however, that many witnesses
highlighted the need for Defence to give close attention to identifying and
mitigating risk during the early stages of the procurement process, especially
during capability definition. In their experience, the consequence of any
failure at this stage of the procurement has the potential to surface later in
the acquisition process and to cause serious disruption to a project.[12]
5.9
Based on international and Australian literature, and with a particular
focus on defence procurement, the committee notes that to be effective, risk
management should or must be:
- considered from the outset or formative stage of a project when
critical decisions are made that have significant implications for the overall
success of an acquisition and its through-life support;[13]
- an iterative process throughout the acquisition and sustainment
of capital equipment involving the identification, analysis, mitigation
planning, mitigation implementation and tracking and reporting of
risk—consulting and communicating with all stakeholders on risk and risk
management is important;[14]
- comprehensive, systematic and applied consistently across the
entire organisation at the enterprise, business and operational level;
- broad-based ensuring that all the various factors associated with
a defence procurement are assessed for risk—'even those considered as obvious
need to be identified and treated'—budget, schedule, technical requirements,
workforce, environmental, infrastructure, contract and stakeholder relations;
- fully integrated and embedded in an organisation's culture so
that risk management policy and practice is part of management thinking and
actions and permeates all levels of the organisation— enterprise level, function
level or business unit level—senior managers in particular must show leadership
and commitment and managers at all levels must take responsibility;[15]
and
- part of a continuous improvement system where experiences in risk
inform revised risk assessment and management strategies—this means that
lessons must be learnt from previous experience and applied to future decisions
and actions regarding risk management.[16]
Committee view
5.10
Clearly, risk management is a part of good governance and not an add-on.
Although the acquisition of major defence assets is a high risk activity, sound
management practices can reduce the potential for poor results. Thus,
responsible for large and complex projects involving cutting edge technology, defence
organisations have a very real interest in managing risks. Failure to do so can
result in poor project performance—cost overruns, schedule slippage or shortfalls
in capability. Thus, it is essential for an organisation to be well placed to
anticipate, understand and manage risk. To do so effectively, it should have a
sound risk management framework that binds all forms of procurement undertaken
by the organisation and be front and centre of decisions for managing its projects
effectively.[17]
And as pointed out elsewhere, good risk management in the defence environment
will occasionally need to tolerate some failure. For example, the airborne
warning and control system where, despite the risk and some failure, a lot of
the capability sought was eventually achieved—though perhaps it could have been
better assessed at the outset.
Risk management in Defence
5.11
In Australia, the Defence Procurement Policy Manual defines risk in the
defence context as being concerned with the 'things that can go wrong' to its
projects and which may prevent the project from being a success. It states that
the government considers that a successful project is one that 'delivers a
fit-for-purpose capability, as approved by Government, within the approved
budget and schedule'.[18]
Policy
5.12
In 2009, Standards Australia published a revised version of its
principles and guidelines on risk management.[19]
The Commonwealth Procurement Guidelines and companion guides such as the
Commonwealth Policy Framework for National Public Private Partnership also
advocate the use of risk management and provide advice on its application. Consistent
with these guidelines, Defence has produced a number of key documents that
further underscore the importance of understanding risk and its effective
management. They include:
-
Defence Procurement Policy Manual;
- Defence Capability Development Handbook;
- Technical Risk Assessment Handbook;
- DMO Project Management Manual; and
- DMO instructions.
5.13
Based on these documents, Defence has certainly demonstrated that it is an
organisation that recognises the importance of risk management as an
indispensible part of effective governance that underpins sound decisions.[20]
For example, the Defence Procurement Policy Manual (the Manual) clearly states Defence's
commitment to 'a comprehensive, coordinated and systematic approach to risk
management'.[21]
It recognises that sound risk management is a vital component of good corporate
governance and that a 'well developed and managed risk management plan will
lead to informed decision-making to ensure the desired result is achieved'.[22]
5.14
Air Marshal Harvey acknowledged that Defence must manage risk. When it
comes to the practical application of Defence's policy on risk, he explained
that Defence has 'a very structured approach' that has been refined in line
with previous reviews and Defence's internal work. He also noted that Defence's
consideration of risk is broad, which covers cost, schedule, capability,
technical, workforce and overall programmatic risk.[23]
Guidelines
5.15
The Defence Capability Development Handbook (the handbook) sets out the
specific steps to be taken with regard to risk management. This document is a
guide to the capability development body of knowledge, best practice and
processes for Defence. It provides directions and offers advice on risk in defence
procurement. According to the handbook:
Command and management processes at all levels are required
to plan, apply, measure, monitor and evaluate the functions an agency performs,
with due cognisance of risk assessment and subsequent risk management.[24]
5.16
It states that 'every proposal must ensure that Government is aware of
the risk it accepts in making an investment decision. Risks must be measured,
mitigated and managed to ensure there is a tolerable risk-return balance'.[25]
5.17
Unlike the procurement policy manual, the handbook does not have a
discreet section on risk management. It deals with the implementation or
practical application of risk management at every phase of the acquisition
process. It recognises the need to consider risk early and for it then to be a logical
and sequential process throughout the capability development cycle.
5.18
The committee examined Defence's Procurement Handbook and related
documents—DMO Project Management Manual and Project Risk Management
Manual. It looked at risk management from the needs phase through to entry to
the DCP, first and second pass approval, acquisition, tendering and contracting
to delivery, including the use of early warning systems designed to stop
projects becoming projects of concern. It considered the various panels and
committees that review the project proposals at milestones during the
acquisition process including the Options Review Committee (now replaced by the
Project Initiation and Review Board), the Capability Gate Review Boards, the Defence
Capability Committee (DCC), the Defence Capability and Investment Committee, Service
Chiefs and Group Heads and finally the Secretary of Defence and CDF who clear a
submission for government consideration and final approval.
5.19
It is clear that, although the committee has not described step by step
Defence's risk management process as set down in its manuals and guidelines, it
found that the contents of the documents align with good practice. For example,
the committee notes the comprehensive coverage Defence gives to risk management
in its policy and practice guidelines. Defence clearly recognises risk
management as:
-
integral to efficiency and effectiveness; and
- a means that enables agencies 'to proactively identify, evaluate
and manage risk, opportunities and issues arising out of procurement related
activities'.[26]
5.20
Consistent with the key principles of sound risk management, the
committee found that the handbook, DMO Project Management Manual and DMO
Instructions recognise the importance of:
- considering risk from the earliest stages of procurement
planning;
- monitoring risk and its treatment on a systematic basis
throughout the procurement process—risk management is treated as a continuing
process, with opportunities to re-evaluate risks at key stages of the
procurement process;
- taking account of all aspects of risk including costs, schedule,
capability, programmatic and workforce; and
-
providing senior leaders and the government with sufficient and
reliable information upon which to make decisions.
5.21
There can be no doubt that Defence's stated policy on risk management
and the guidelines and handbooks intended to assist officers implement the
policy is consistent with international and Australian standards.
Conclusion
5.22
In light of Defence's risk management policy, the practical guidance
provided in its procurement handbook and the in-built review structures; it
would appear reasonable to assume that risk management is a prominent and
essential element of Defence's procurement culture. If implemented properly, Defence's
policy, supporting documents and practices should work effectively to mitigate
risks. Evidence, however, suggests otherwise. Indeed, the poor performance of
some major projects, detailed in chapter 2, indicates that risk management may
not have been as robust as it should have been—for example, cautionary advice
from domain experts not understood, downplayed, misplaced or ignored as it
moves up the decision-making hierarchy. On countless occasions, the ANAO has
noted that this repeated failure to identify or acknowledge risk is simply a
manifestation of bad management in an unaccountable system.
5.23
In the following chapter, the committee begins its examination of the underlying
causes of poor performance.
Navigation: Previous Page | Contents | Next Page