Chapter 4
Challenges for defence procurement
4.1
Defence projects for the acquisition of major capital equipment face an
array of internal and external forces and influences that generate uncertainty
about the extent to which they will meet their objectives.[1]
High levels of unpredictability give rise to risk.[2]
In this chapter, the committee examines the main challenges to the success of
an acquisition project. It looks at the sources of uncertainty relating to the
procurement of major capital equipment both in a world-wide context and more
specifically in Australia. In this context, the committee looks at recent
literature on risk management in defence procurement and its emphasis on the central
role that such practices have in successful acquisition.
Large, complex and costly undertakings
4.2
The acquisition of major defence capital assets is of a scale and
complexity that presents 'formidable and ever-increasing challenges'.[3]
The recent UK Ministry of Defence's strategy for procurement reform noted that
'acquisition isn't easy'.
It involves running large numbers of projects, many of them
big and complex. Many are also at the leading edge of technology and
innovation.[4]
4.3
Australia's experience is no exception. Indeed, according to a study by
the Helmsman Institute, defence projects in Australia are of a level higher in
complexity than projects in Australian organisations in other sectors.[5]
It found further that defence projects are not only more complex but that
generally Defence is managing a higher number of these complex projects during
any given period compared to others in Australia:
Most corporate and government organisations may have one or
two 'Organisationally Complex' project[s] underway at any one time, and once
every five to ten years a sector may have a 'Nationally Complex' level project.
Defence will have numerous 'Organisationally Complex'
projects, several 'Nationally Complex' projects in any year, and may start one
that is 'Nationally Significant' every ten to fifteen years.[6]
4.4
This degree of complexity is part of a continuing trend which is
expected to increase. The study found:
For some of the more complex projects, such as the Air Warfare
Destroyer, LHD and the new Submarine, the projects are in early phases and will
continue to be a focus of Defence over the next planning horizon.[7]
4.5
Thus, complexity is unavoidable and risk inherent in any major Defence
acquisition project.[8]
Technology is a key source of complexity.
Advances in technology and the
importance of integration
4.6
The central role of technology and the constant quest for improvements pose
significant challenges for defence procurement.[9]
The 2009 White Paper enunciated the government's objective of developing and
maintaining a capability edge. It stated that, 'giving our forces a capability
advantage is both desirable and necessary if it prevents conflict, or allows us
to prevail in conflict, and minimises our casualties and materiel losses'.[10]
The paper reasoned that military modernisation, particularly in the
Asia-Pacific region, and the proliferation of advanced military technologies
will mean that Australia's ability to maintain a capability advantage will come
under increasing pressure.[11]
It recognised the need for Australia to maintain its necessary strategic
capability advantage:
Superiority in combat and other forms of military operations
will hinge on continual advances in military technology...[12]
4.7
Sonartech Atlas noted that the rate of change in technology can be an
issue in itself when trying to determine complexity, maturity and risk.[13]
Dr Richard Brabin-Smith also observed that science and technology in defence-relevant
fields continues to develop, 'often at breath-taking speeds, especially in
anything that is touched by electronics and computing'.[14]
Witnesses also noted the importance to Australia's national interests of maintaining
a technological advantage at least in the areas of capability that are central
to Australia's security. According to Dr Brabin-Smith, if it is accepted that
Australia's strategic circumstances will become more demanding, Australia needs
'to ensure that Defence is close (or at least closer) to the leading edge of
what is technically or technologically achievable—both at the time of
acquisition and through in-service upgrades'.[15]
4.8
As a consequence, Defence will look to acquire capabilities that provide
a competitive edge by anticipating tomorrow's technology. Sometimes Defence try
to do this all at once with multiple 'new' and untried elements rather than
incremental change. Keeping pace with these rapid advances adds yet another
layer of complexity for Defence and the uncertainty of future advances increases
the risk of 'things going wrong'.
Integration
4.9
At a time of rapid advances in technology, newly acquired assets or
upgrades are intended to keep the nation's capabilities at the forefront of
such developments. In this environment, integration presents significant
challenges. Incorporating or assimilating sophisticated equipment into larger
systems increases the degree of complexity. Mr Bruce Green, seven years as
Deputy Secretary of Defence (Acquisition) for New Zealand, noted that large
expensive military capabilities are a mixture of complex systems that need to come
together to deliver the desired outcome. He explained that they can take years 'to
develop and mature to a point where there is confidence that on any given day
it is going to function as required'.[16]
4.10
Defence cannot escape this trend toward increasing complexity. According
to one industry representative, Australia is 'increasing the net centricity' or
interconnectedness of its capabilities. He was of the view that Australia is 'rapidly
heading down the path where almost everything on the battlefield must be
properly interfaced or integrated'.[17]
He estimated that 39 other programs in the DCP are connected in some way to the
LHD's capability ranging from helicopter through to communications, command and
control, and medical equipment projects.[18]
The need to have joined up capabilities of projects that are themselves highly
sophisticated adds to the complexity and increases risk of problems emerging. A
chartered professional engineer with over 30 years experience in project management
observed:
...increased risk is frequently reflected in the number of
issues that arise when new command and support system capabilities, often
associated with different platforms, are brought together and expected to work
seamlessly. It is unresolved interface issues and the delays caused by the
increased complexity of defence projects that have largely led to criticism of
the management and governance in defence projects.[19]
4.11
DSTO concurred with the view that systems integration for defence
projects—having them knit together smoothly—is a major challenge for both
industry and Defence.[20]
It should be noted, however, that on occasion Defence unnecessarily compounds
things. The integration of MU90 on the AP3 was achievable as suitable interface
'boxes' existed. Defence decided to set its target being a joint integration of
the Joint Air-to-Surface Standoff Missile (JASSM) as well as MU90. The delays
to selecting JASSM and compounding impacts of having multiple suppliers and
regulators involved made the task almost impossible and it was finally
cancelled for the AP3.[21]
Changing geo-strategic environment
4.12
Advances in technology are also taking place in a world of shifting
geo-political and strategic situations. For example, the 2009 White Paper
stated:
The ADF will be required to operate in an environment which
is increasingly complex, as more potential adversaries will have access to a
wider range of capabilities which are comparable to ours, or will be able to
exploit vulnerabilities in ours.[22]
4.13
The principal drafter of the White Paper, Mr Michael Pezzullo, explained
that in preparing the paper, the authors had to be aware of change over time—strategic
environment, fiscal circumstances and the nature of technology.[23]
Dr Brabin-Smith similarly referred to the imponderables of potential changes to
geo-strategic circumstances over which Australia itself has little direct
control. Australia also has no influence over the way emerging technology might
alter aspects of warfare.[24]
In his view:
These factors imply that, at least in some areas of defence
capability, there will necessarily be greater technical and therefore
acquisition risk: cost, schedule, and perhaps the level of capability
that in practice proves to be achievable.[25]
4.14
Thus, Defence is required to make decisions about future acquisitions
and the upgrades of existing assets without full knowledge of the threats or budgetary
constraints it is likely to face. This uncertainty is yet another source of
risk to the successful performance of a Defence major acquisition project. Dr
Brabin-Smith argued that as a consequence of this uncertainty there is 'a clear
need for a robust and thorough approach to risk management: at the conceptual
phase; during procurement; and in through-life support'.[26]
Conspiracy of optimism
4.15
In this environment, defence organisations, with an eye to future
developments and striving to maintain a technological edge, are drawn naturally
to the latest in technology and to what might be possible. Mr Bruce Green suggested
that contractors can at times exaggerate their ability to deliver
complex systems. [27]
4.16
Unwarranted confidence in the ability of industry to produce a
capability is not, however, confined to the defence industry. Mr Green observed
that an acquisition entity, without understanding the risks, can allow 'good
ideas' for capability enhancement to become part of the procurement.[28]
In this regard, Mr King referred to a culture in the whole Western world, whereby
defence organisations and the nation want the latest capability they can get,
as soon as possible and at the least expense. He explained that industry
sometimes inadvertently feeds that desire by suggesting that very advanced
technologies may come sooner than is really practical—a situation described as
a conspiracy of optimism.[29]
4.17
The temptation to overreach may be further encouraged by the
predisposition to 'gold-plate' the operational requirements.[30]
The Australian Association for Maritime Affairs attributed this tendency to the
view that 'there probably won't be another similar building program for up to
20 years'.[31]
Australia is not spared from this over confidence in industry's ability to
deliver unproven capability and Mr King stated that 'we need to do as much as
we can to stop it'.[32]
4.18
There are numerous safeguards against over optimism including having
suitably qualified and experienced people critically analyse a proposed
solution and identify risks associated with its acquisition. Another measure is
to engage experienced project hardened individuals to review a project at
critical stages of its development. These are discussed later in the report.
Long term ventures and extended
timeframes
4.19
Moreover, decisions about these highly complex acquisition projects take
place not only in a dynamic and uncertain technical and strategic environment
but over an extended period. The time it takes from identifying a capability
need or deficiency to when a decision is made on procurement and actual
delivery spans many years.[33]
As an example, Defence referred to project SEA 4000—Air Warfare Destroyer worth
over $8 billion—which took Defence around six years to develop the proposal.
4.20
In an age where technology is constantly changing, this long-term
process, from identifying a capability need to developing and delivering it, means
that science and technology priorities may no longer be current at the time of
completion. Babcock stated that, given the complexity of defence projects, it
is inevitable that definition of the contract deliverables will evolve over the
project's duration.[34]
In this regard, another company, Sonartech Atlas, cited changes in computer
software which can have substantial effects on procurement outputs and outcomes
over longer term schedules. It noted:
When considered against the 10 year procurement lifecycle,
the underlying technology may have undergone four possibly five iterations,
from the time the project was included in the DCP up until the Government
approves it at 2nd Pass, let alone introduction into service.[35]
4.21
While in many cases advances in technology warrant changes to initial
plans, there is also the increased risk of 'scope creep'. For example, Mr Green
noted that a contractor or the military may suggest 'some new and emerging
technology that may be a useful enhancement to the capability'. In his view,
sometimes the change may be 'unavoidable' or simply a case where 'a component
of a system may be a new model with enhanced performance'. He suggested that in
this situation there are risks 'especially if the new model has not been fully
tested or integrated into like capabilities'.[36]
4.22
ANAO also noted the extended timeframe for procurement of major Defence
capital equipment projects and the uncertainty generated by reforms that are
implemented after a project has started. It cited the Lightweight Torpedo
project which began just after the 1997 Defence Efficiency Review and was
managed by the Defence Acquisition Organisation and then its successor, DMO.
The project's management and review arrangements passed through the formation
of Systems Program Offices (SPOs) and different phases of the project were
subject to different approval processes, the most recent phases passing through
the post-Kinnaird Review strengthened two-pass approval process.[37]
4.23
Sonartech Atlas explained that longer schedules for acquisition can have
a marked influence on the efficiency and effectiveness of the procurement
activities, or 'more precisely the longer the exposure the greater the
likelihood of an issue'.[38]
The long lead time for complex projects also has implications for staff
continuity with extended project schedules inevitably resulting in staff
turnover which can disrupt the project's progression and, in some cases, its
direction.[39]
4.24
In this context of timeframes, defence organisations must also consider
the through life operation and sustainment of an acquisition. The Australian
Association for Maritime Affairs noted that an operational capability that has
taken many years to deliver may then have to remain in service for a further
20-30 years. It observed further that throughout this period 'the surrounding
operational environment continues to evolve'. The Association also pointed out
that government consideration takes place in a changing domestic political,
foreign policy and economic environment where scientific and technological
developments continue at a rapid pace.[40]
4.25
Clearly, as the length of an acquisition period grows the more difficult
it can be to define and manage the procurement activities. Rapid advances in
technology, staff turnover, reforms to the procurement process and shifting
government policy and funding priorities add to the complexity of defence
acquisition. There are ways of managing these extended timeframes such as
having and adhering to specified timelines and striving for incremental
increases in capability with lower complexity, risk and cost.
Self-sufficiency for Australia as a
medium sized country
4.26
An important consideration that relates to Australia's security
interests is the extent to which the nation should be self-reliant in providing
for its own defence capabilities. The 2009 White Paper recognised that total
self-sufficiency in defence industries would be impractical for a state the size
of Australia. The government, however, has stated its commitment to ensuring
that certain strategic capabilities remain resident in Australia.[41]
Thus, not only does Australia seek to be at the forefront of technology but
must decide whether it wants to be self sufficient in the design, or build or
maintenance and upgrade of a particular capability. The White Paper stated that
in the current environment, 'Australia's self-reliant scientific and
technological capabilities will become a relatively more critical element of
our strategic capability advantage...'[42]
4.27
A critical and important decision for government is to identify the
areas in which it wants the country to be self-reliant. In its report on Naval
shipbuilding and repair, the committee noted the challenges for Australia in
endeavouring to reconcile its desire for self-sufficiency in areas deemed to be
a national security priority with the practical limitations imposed by cost and
technology.[43]
Furthermore, the Helmsman Institute referred to the government's policy on
local manufacture. It noted that, in order to deliver the capability platform,
many projects create local manufacturing capability which increases project
complexity, in many cases 'quite significantly'.[44]
The challenge then is for government to ensure that Australia with its small
defence industry and limited budget has, at hand and in country, the skills and
resources deemed essential to secure the national interest. The underestimation
of industry's capacity in the Melbourne shipyards for the AWD is such an
example, as is the predicted skills needed for the new submarines (see
paragraph 13.7–13.10).
Marketplace developments
4.28
Largely outside Defence's control, the global defence market is another
source of risk to a defence acquisition program. The increasing demands for
improved capability at an affordable cost have produced dramatic shifts in the
industry. For example, in many maritime nations, there has been a trend over
many years towards consolidation from a larger industrial base with shipbuilders
amalgamating to a few in number. The same cost pressures have resulted in
increasing mutual interdependence among prime companies and also among major
sub-prime contractors.[45]
The committee found in 2006 that:
This trend toward business consolidations, partnerships and
alliances cuts across industries and national borders as countries are finding
that, especially with highly complex systems, they cannot be self-sufficient in
all aspects of a ship's design and construction.[46]
4.29
In its submission, Defence noted that its procurement takes place in a
constrained marketplace which 'is changing in important ways that will impact
future equipment acquisitions'. It explained:
Australia's major allies are increasingly developing single
lines of development for complex platforms through spiral acquisition processes
that require very early Australian engagement if our specific needs are to be
taken into account. Highly complex and integrated weapons systems such as the
F-35 fighter aircraft cannot be purchased and then developed to suit Australian
needs within reasonable cost or risk parameters and there is no other suitable
fifth generation fighter to choose from. While providing opportunities for
Defence to be involved in the early stages of major new allied capabilities,
this type of international acquisition process limits choice, and limits our
ability to influence cost and the timing of equipment delivery.[47]
4.30
As mentioned above, this situation is compounded by 'substantial
consolidation in the global defence industrial base since the Cold War'.
Defence observed that as the majority of Australian defence companies are
subsidiaries of major foreign defence suppliers, Australia's defence industrial
base is caught up in this international trend.[48]
4.31
These developments create significant challenges especially for Defence's
ability to retain skill sets required to be a smart customer.
Managing risk
4.32
Overall, defence acquisition is informed by a complex and changing
strategic environment with key decision-makers keeping an ever-watchful eye on
likely future developments.[49]
This dynamic context, coupled with acquisition projects that span lengthy
timescales, means that at the time a capability enters into service, it may no
longer meet strategic imperatives.[50]
Moreover, the current environment continues to present major difficulties for
defence procurement—evolving requirements, increased emphasis on systems
integration, globalisation, prolonged life cycles and rapid advances in
technology.[51]
Drawing together a number of the factors that can influence the performance of
a defence acquisition project, the Helmsman Institute found that given the
trends that underlie the current complexity; it would 'comfortably predict that
the future projects will increase in complexity'.[52]
Self-inflicted complexity
4.33
Clearly, there are many sources of often-unpredictable change that
generate significant risks, particularly for large and complex defence
acquisitions.[53]
Indeed, the risks to the successful procurement of major defence assets are considerable
and wide-ranging. Some countries, however, have recognised that projects for
acquiring major capital equipment not only fall short in meeting those
challenges but that their own practices add to or compound the problems. Unrealistic
requirements, a lack of early systems engineering, acceptance of unreliable
estimates based on overly optimistic assumptions about costs and timelines and
the failure to commit adequate funding and poor contract management all
contribute to poor outcomes.[54]
A breakdown in just one area of a major capital procurement can have serious implications
for the success of the project. The US Government Accountability Office (GAO)
noted that unwarranted risks can undermine an acquisition in a number of ways.
It found:
A poorly conceived acquisition is doomed from the outset,
while a poor contract selection or an inadequate workforce can weaken the
government's ability to manage and oversee the acquisition. Therein lies the
challenge: it takes many things for an acquisition to succeed, while only one
source of unmanaged risk can cause a poor outcome.[55]
4.34
Many defence organisations worldwide acknowledge their own failings in
their major acquisition projects. For example, the Canadian Auditor General
found that for two major projects—the purchase of the Cyclone and the Chinook
helicopters—National Defence had underestimated and understated the complexity
and developmental nature of the intended aircraft. Further it had not developed
full life-cycle plans and costs in a complete or timely way and had not fully
complied with the oversight and approval framework established in its Project
Approval Guide.[56]
The United States (US) has also raised concerns about its procurement
performance. The 2010 US Quadrennial Defense Review Report recognised
that shortcomings in Defence's acquisition process placed the Department 'at
risk of being unable to deliver the capabilities it needs, when it needs them,
and at acceptable costs'.[57]
Noting the importance of 'a healthy acquisition process', it stated:
The Department and the nation can no longer afford the
quixotic pursuit of high-tech perfection that incurs unacceptable cost and
risk. Nor can the Department afford to chase requirements that shift or
continue to increase throughout a program's life cycle.[58]
...we must not embark on programs with artificially low cost
estimates, immature designs and technology, fluid requirements, excessive
technical authority certification requirements, unstable budgets, and
unsustainable procurement profiles.[59]
4.35
The US Department of Defense noted that over recent decades and across
multiple administrations, the Pentagon’s acquisition system had developed major
problems that hampered its ability to acquire critical platforms and capabilities
in a timely manner and at acceptable cost. They include:
- the requirements for new systems are too often set at the far
limit of current technological boundaries;
- the Pentagon’s acquisition workforce had been allowed to atrophy,
exacerbating a decline in the critical skills necessary for effective
oversight; and
- the system of defining requirements and developing capability too
often encourages reliance on overly optimistic cost estimates.[60]
4.36
In many ways, the Australian Defence organisation is susceptible to the
same shortcomings as its counterparts.[61]
The decision to integrate the MU90 torpedo onto the AP-3 Orion combined with
JASSM integration is a notable example. The Helmsman Institute found that
Australia's defence's projects were not only highly complex but a number of them
indicated that 'Australia had taken on development challenges for solutions
that more other defence forces had either given up on or had failed to deliver'.[62]
The report found:
While most observers interviewed understood the need to
invest in solutions that would stand the test of time (in some cases a
considerable life span), a number of projects raised concerns that the
complexity was so high that the project was placed at risk of never delivering
the required capability, and failing to work. Some projects had to re-architect
the solution midway into development at considerable expense and delay.[63]
4.37
Indeed, in chapter 2 the committee detailed many of Defence's projects
that have experienced self-inflicted problems—inadequate planning and scoping
of the project, failure to appreciate the developmental nature of the project
or complexity with integration, poor project management, underestimation of
defence industry capacity, lack of skilled workforce, inadequate contracting
arrangements, insufficient consideration of through-life support, and a
breakdown in the relationship between the relevant service, DMO and the
contractors. Organisationally, all these factors are fundamentally important. As
one witness observed, 'In short, Defence/DMO have been able to keep Navy tied
up in port to an extent not achieved by any enemy force'.[64]
4.38
The recurring nature of the shortcomings is particularly concerning and
suggests that lessons from previous troubled projects are yet to be learnt and
that inadequate risk management may be at the heart of the problem. In the
following chapters, the committee seeks to understand the factors behind poor
performance and what can be done to change behaviour rather than process. To do
so, the committee explores the fundamental components underpinning good
governance and sound decision-making—risk management, adherence to policy and
guidelines, accountability, contestability and the skills and experience of
those engaged in defence acquisition including the quality of their analysis.
Navigation: Previous Page | Contents | Next Page