...it takes many things for an acquisition to succeed, while
only one source of unmanaged risk can cause a poor outcome.
This chapter draws together the evidence presented in the previous
chapters in order to answer one of the key questions driving the committee's
inquiry—whether entrenched structural impediments to efficient and effective
leadership within Defence are at the source of Defence's procurement woes. In
this report, the committee has sought to establish whether a reallocation and
redefinition of roles, functions and responsibilities is required. Indeed,
whether the current management matrix needs to be overhauled or even
Challenges and Defence's responses
The committee notes and supports evidence that highlights the
competence, dedication and hard work of people at all levels of Defence's
Defence projects for acquiring major capital equipment face an array of
internal and external forces and influences that create significant difficulties
for Defence. In fact, such projects are of a scale and complexity that they present
'formidable and ever-increasing challenges'.
The problems identified in defence procurement, however, are largely a function
of the Defence organisation's own making—unintentionally self-inflicted. They
include: inadequate planning and scoping of project; poor risk management from
beginning to end of project; failure to appreciate the developmental nature of
the project or complexity with integration; poor project management; underestimation
of defence industry capacity; lack of skilled workforce; inadequate contracting
arrangements; insufficient consideration of through-life support; and a
breakdown in the relationship between the relevant service, DMO and contractors.
The committee finds that the current management structure in Defence has
produced an organisation that lacks a robust risk regime: an organisation where
its personnel are insensitive or unresponsive to risk, where no one owns risk
and is incapable of learning lessons from past mistakes. In brief, Defence is currently an organisation
that cannot anticipate, understand or manage risk—a fundamental flaw in an
organisation that undertakes large-scale and complex projects that are in
essence engineering operations. Importantly, this failure to own risk and to
learn lessons is not a process problem—it is clearly a weakness deep within the
organisation that permeates outwards and effectively precludes people from
taking responsibility and being accountable.
The inability of Defence to learn from past mishaps is a particularly
salient point. Defence may well argue that the failures noted in this report
are drawn from history: but if it cannot or will not apply lessons from
previous projects to current and future ones then it is destined to repeat
them. Learning lessons is not only about keeping a risk register or a data
base—that is simply a process—it about those lessons becoming part of the
Process versus genuine reform
Defence's responses to the evident failings in their procurement
projects have tended to focus on process. Even its most recent initiatives to
clarify responsibilities and strengthen accountability look to process for
solutions—re-badging and improving the Initiation and Project Review Boards; emphasising
the role of joint project directives, enforcing Materiel Acquisition Agreements
(MAAs); and introducing project charters (Mortimer recommendation). If
implemented and properly adhered to, such measures should go some way to reduce
the opportunities for a project to slip off the tracks. The committee is
concerned, however, that such measures merely promote form over substance and it
remains to be convinced that in practice they are effective.
The committee has highlighted prevailing practices and circumstances in
Defence that sabotage its endeavours to realise the objectives of these
initiatives. They include non-compliance with policy and guidelines, an
environment that has generated multiple and confusing layers of bureaucracy, poor
linkages between key agencies, and a lack of, or mismatch of, appropriate
skills. Thus, despite Defence's confidence in its initiatives, the committee envisages
that, with the passage of time, the damaging behaviours, which have simply been
papered over, will again surface to perpetuate the pattern of poor performance.
For example, the committee fears the potential for the Project Initiation and Review
Board to turn out to be a simple re-packaging of the Options Review Board and
hence replicate the same shortcomings—an unwieldy committee made up of a number
of groups lacking authority and whose members do not have the required
experience or specific competence for the task. The committee has heard nothing
to indicate that, despite current enthusiasm for the boards, they will not
revert to form.
Indeed, the weight of evidence indicates that not only has Defence's
preoccupation with process been misguided but it has been counterproductive. In
response to identified problems, Defence has created a procurement process that
is convoluted and overburdened by administration. The committee is of the view
that the entire organisational structure of Defence must be simplified and
streamlined. Only by reducing the number of stakeholders (groups) involved in
the process can the excessive administrative burdens and committees be reduced.
Defence is also convinced that it has a robust quality assurance
framework with the revamped gate reviews and the independence of the DSTO, DMO
and CIR Div. It is of the view that these independent bodies provide the
necessary contestability to ensure that decision-makers are provided with a
range of well-considered, impartial and specialist advice.
In this regard, the committee acknowledges that Defence has a quality
assurance framework that is designed to provide internal contestability and
external scrutiny. But again, the committee finds that the overall impression
of Defence as an organisation with a healthy and open approach to independent
review and diversity of views fails to match facts.
All sectors, including industry and defence analysts, supported the work
of the gate review boards. A very worrying development noted in the ANAO's
audit of gate review boards, however, was the growing tendency for a manager
with a direct connection to the project to be appointed chair of the review.
According to the ANAO this trend has increased in recent times:
During the first year DMO conducted Gate Reviews (July
2009–June 2010), 33 per cent of Gate Reviews of ACAT I and II projects were
chaired by a manager with some responsibility or accountability for the project
under review. During the second year (July 2010–June 2011) this increased to 42
per cent. During the first six months of IPPO's management of all Gate Reviews
this increased further to 50 per cent.
Throughout this report, the committee has referred to numerous instances
of non-compliance with policy or guidelines. The gate review examples cited by
the ANAO throw into sharp relief, how genuine, sound reforms can be rendered
useless by a management structure that cannot or will not exert authority. This
latest clear disregard of policy whereby the independence of gate review chairs
was compromised underlines the committee's scepticism about the effectiveness
of other recent initiatives such as project charters, MAAs, and the project
initiation and review board.
In the committee's view, Defence have been tinkering at the margins of
the problem, giving the impression that by improving process, the desired change
in behaviour will follow. The committee believes that such an approach only
serves to mask fundamental weakness in the overall management structure of
Defence and its major acquisition programs. Thus, despite a raft of reforms and
reliance on Defence quality assurance frameworks, the persistent pattern of
poor project performance continues. Problems such as mistaking a developmental
project for a genuine off-the-shelf product indicates that this internal filter
and the gate reviews have not worked as well as they should. Indeed, the
reforms have done nothing to prevent highly developmental projects being submitted
to the Chief of the Defence Force and the Secretary of the Department and
ultimately to government as off-the-shelf products. Clearly, the answer is not
With regard to the independent advice provided by agencies such as DSTO
and DMO, and advice obtained (or not) from domain experts and industry, the
committee again finds Defence's depiction of their effectiveness at odds with
reality. The committee cited observations about DSTO's advice not receiving the
respect it deserves, of Defence agencies generally undervaluing technical
advice and, in some major projects, downplaying, misinterpreting or even
completely disregarding specialist domain advice such as pre-contract T&E
The committee also noted Defence's response to Mortimer's recommendation
that the CEO DMO 'should provide independent advice to Government on the cost,
schedule, risk and commercial aspects of all major capital equipment
acquisitions'. According to Mortimer, the CEO DMO should also be a permanently
invited adviser to government committees considering defence procurement.
Defence made clear that the views of organisations including CDG, DSTO, and
capability managers 'must be properly reflected in the cabinet submissions'.
It stated further, however, that 'it would not be appropriate for DMO to make
coordinating comments on Defence cabinet submissions because, for procurement
matters, DMO is intimately involved in preparing these submissions'. Defence's approach
contradicts the Mortimer principle and effectively negates one element of
contestability which relies on independence for its effectiveness.
It is also important for agencies' advice and recommendations to be
clearly discernible so that they can be held accountable for them. But Defence's
'one view', mantra effectively removes diversity of opinion provided by
specialist agencies, experts and senior Defence leaders by presenting just one position.
A number of witnesses acknowledged that it was appropriate for Defence to speak
with one voice 'provided that what that one voice was saying had been arrived
at via a process of thorough contestability and lots of frank and fearless
advice, carefully listened to within Defence'.
Unfortunately such is not always the case in Defence.
Disenfranchised capability managers
The most glaring consequence of Defence's failure to effect meaningful
reforms has been the disenfranchisement of the capability managers. Capability
managers ultimately operate the equipment or platform being acquired and are
responsible for ensuring that the acquisition is fit-for-purpose In 2003, Mr
Kinnaird argued that:
Capability managers, the most prominent being the Service
Chiefs, should be made responsible and accountable for monitoring and reporting
to government on all aspects of approved defence capabilities.
This responsibility would be for 'the whole of capability from the point
where government approves a particular capability option, that is at second
pass approval, through to the time that the capability is retired from service'.
Mortimer also made a number of recommendations that, if implemented properly,
would make capability managers an integral and engaged part of the acquisition
process. This would include capability managers reporting regularly to
government 'on the status of the capability development initiatives for which
they are accountable'.
Yet evidence before the committee is unequivocal—capability managers
have been left out of the acquisition loop. For example, Mr King accepted that
there was a time post Kinnaird where the centralisation of the capability
development under CDG and the DMO operating as the acquisition organisation
'appeared to disenfranchise the capability managers in the process'.
He stated that the situation led to 'a period where, despite having the two pass
process in place, the CM, CDG and DMO were not interacting, coordinating and
integrating as well as they might'. This breakdown in communication was
particularly evident in the maritime space. Mr King explained in simplified
terms what he thought had happened:
...the customer base―the capability manager―had
developed a feeling that DMO would just pass something or throw something over
the fence at them and they would have to take it. I think they had fallen into
a mode of 'Well, I'll see if I like it when I get it.'
It is clear that capability managers have much ground to recover and must
regain authority over key areas of capability development, particularly the
responsibility for determining the technical specifications they require for
acceptance into service. Most notably, this applies to the Chief of Navy.
Capability managers must also have adequate and appropriate resources,
including a core of trained professional engineers, in order to exercise their
responsibilities. They must also be held to account for the way in which they
exercise that authority, which means that their decisions must be traceable
back to them.
Defence believes that capability managers are now 'upfront' and cite the
fact that they currently sign the MAA as evidence of that engagement. The
committee has already expressed its doubts about the effectiveness of this
measure. It should be noted that MAAs have been in place since 2005, yet they
have failed to do their job. An important issue for the committee is how to
prevent a situation developing that effectively disenfranchises the capability
manager from the acquisition process.
In this report, the committee has shown repeatedly that while Defence
has correct practices and procedures on paper, it fails to implement them properly.
More tinkering of manuals and guidelines and policy statements and adding more
process to an already overburdened one will not work. If not accompanied by
genuine changes in management, such initiatives will simply compound Defence's
problems. In this regard, Defence needs to pay close attention to creating an
environment, especially through its management structure, that is inclusive,
counters the tendency for groups to work in silos and allows those with
responsibility to exercise their authority. In doing so, Defence should also be
intent on removing layers of administration not adding to them.
The committee is recommending a restructuring of Defence that would
ensure capability managers have a central role in the acquisition and
sustainment of their major capital equipment. The intention would be to institute
direct contractual agreements after second pass between clients (capability
managers) and contracted providers with no third party involvement. Without
such a standard commercial approach, there will be no change, only more process
and more red tape clogging up the system.
Continuing struggle for skilled
The committee also believes that by focusing on process to solve
procurement problems, attention may be diverted from the more important matter
of finding the right people for the right position so they can drive necessary
change or implement process more effectively and efficiently. One industry
...Organisational structures only go part way towards solving
performance issues...I could have any organisation structure I like that aids
communication and interaction. If [we] do not have the right people with the
right competencies and the right way of behaviours, then the organisational
structure is worth nothing.
Throughout the report, the committee has underlined the futility of Defence
resorting to more information, more process, more people and committees or
paperwork rather than having 'good-quality, appropriately qualified and current
staff in correct positions'.
If Defence wants to be a smart customer then it needs knowledgeable people with
a deep understanding of what it intends to buy and highly skilled experts, who
understand commercial realities and are able to negotiate with contractors to
deliver value for money for the Commonwealth. For example, a number of
witnesses referred to inexperienced and inadequately skilled project managers
in DMO. In this regard, a representative from one of the prime contractors
observed that a few of the project managers with whom he had worked were good
people but 'way out of their depth trying to manage a project of which they had
little experience'—'a recipe for disaster'.
Too often uniformed people, with operational experience and technical
knowhow, are engaged as desk officers in general project management, costing of
proposals and administrative tasks. They feel undertrained and ill-suited for
the tasks at hand and moreover they tend to be on short term postings of less
than three years. Clearly, it is important when seconding military people to CDG
and DMO, where relevant, that they are placed where their skills and experience
can be best utilised. A three year posting, or less, in a managerial position
for uniformed personnel is an inefficient use of otherwise very skilled and
experienced people. The emphasis must be on finding the right people and placing
them in the right position.
The critical shortage of engineers and allied technical skills is a
matter that requires immediate and serious attention. While there are many
external forces undermining Defence's efforts to attract and retain skilled
engineers and technicians, the committee is of the view that it is imperative
for Defence to grow its engineering and allied skills base. Indeed, many
witnesses indicated the skills shortage in Defence must be addressed as a
priority: that 'the work on retaining and attracting key personnel cannot wait
In the committee's view, Defence requires a far more targeted and concerted effort
to build up a critical core of skills within its major acquisition groups and
agencies. This also requires the creation of opportunities to gain and maintain
The committee is also concerned about competition between the groups in
Defence involved in procurement and sustainment for skilled personnel,
particularly in the engineering and technical areas, where the supply is
already under pressure from demands from the private sector. The committee
proposes a model for the consolidation of technical skills into each of
the Services, which should address this waste. In this new organisational
arrangement, capability managers would be responsible for the primary technical
input to all capability proposals, test and evaluation in line with central
policy, and all operational and sustainment management. This applies especially
in respect to large and complex single service capability, most notably in Air
Force and Navy. In this way, the committee believes it should be possible to:
- minimise the inefficiency caused by intra-organisational postings
- enable capability managers to rebuild their design engineering,
logistics and technical skill base capable of understanding the most
sophisticated levels of modern defence technology and effectively balancing operational
and sustainment considerations;
- provide meaningful and rewarding skill paths for technically
skilled personnel whether they be uniform or civilian, noting that stability
and continuity of skill may be more achievable from the latter;
provide complementarity of skills rather than the current
- retain skilled staff on long term projects from conceptual
development through to sustainment and disposal from within one organisation,
fully and singly accountable;
- provide a stronger technical counter to industry in contract
negotiations and management, and
- establish greater permanence to Defence's capacity to follow
rapidly escalating technical complexity of defence capabilities around the
Equally, DMO with its reduced role should be better able to concentrate
on building the high level skills needed for tendering, contracting and project
management (whether through recruitment or contracting from industry). These
skills are critical to support the capability managers who will now be
responsible for the acquisition of capability. DMO as the centre of excellence
would also have an independent role in assuring the quality of information
going to government for initial purchase decisions.
Accepting that the Services role will be expanded and DMO's role changed,
the committee recognises the implications of this model for capability planning
in Defence Strategy Group and CDG. With regard to contestability, the planning
and reporting arrangements will need to ensure that the independent voices of
DMO, DSTO and expert agencies such as T&E can be heard by key decision-makers
without fear or favour.
Overall, the committee found that Defence is an organisation that has:
- a growing disconnect between strategic guidance and capability
development with the current foundation document—the 2009 White Paper—setting
an unrealistic and unachievable acquisition program for the ADF's future
- a culture of non-compliance with policy guidelines and practice manuals;
where personnel get 'bogged down' with too much paper work, produce a 'certain
amount of nugatory work' and 'miss the important things going on';
- confused and uncertain lines of responsibility and accountability
that are too diffuse to be effective—the organisation is unable or unwilling to
hold people to account;
- a poor alignment of responsibility due to the excessive number of
groups and agency functions, which gives rise to unhealthy management and organisational
relationships—for example capability managers removed from active participation
in an acquisition;
- weak compliance assurance with and between agencies or groups creating
an environment where, for example, DSTO assessments or technical advice from
domain experts can be undervalued, or even discarded without checks and
balances to make sure dissenting voices are heard by decision-makers; and
- a poor understanding of the commercial world and as a consequence
Defence is yet to engage actively with industry as a collaborative partner in capability
development and acquisition.
The challenge for Defence is to change an organisational structure with
entrenched attitudes that despite repeated reforms:
- cannot learn lessons from past mistakes;
- still resorts to changes to process rather than implementing genuine
organisational reforms designed to clarify responsibilities and make individuals
accountable for their decisions and performance;
- has effectively disenfranchised the end users—capability
managers—who have been left on the sidelines, without authority over key areas
of capability, procurement and sustainment;
- disempowers project managers, most of whom are diligent and hardworking,
but, without clear lines of delegated authority, are unable or unwilling to
make decisions and take responsibility;
- fails to understand and appreciate the importance of
contestability and simply cannot, or refuses to, comprehend the meaning of 'independent
- despite having a 'One Defence' view does not operate as an
integrated enterprise but rather remains an organisation composed of separate groups
working to their own agendas;
- struggles to attract and retain people with the required level of
skill and experience to support acquisition activities including negotiating
with business, tendering and contracting, and particularly engineering, having
over the past 15 years or more allowed its mainstay skills to atrophy
especially the hollowing out of technical skills in Navy;
- undervalues technical advice and has serious shortcomings in
technical analysis, critical to engineering based projects; particularly its
downgrading of the importance of T&E; and
- is yet to achieve the status of smart or intelligent customer and,
for some major projects, has a troubled history with its suppliers—poor
tendering and contracting practices and overall lack of business acumen.
Need for structural reform
The recommendations in this report take account of Defence's attempts to
remedy shortcomings. They also recognise that Defence has made these efforts while
simultaneously attempting to comply with multiple reform agendas arising from a
string of government reviews and directives. The central overriding
recommendations, however, underscore the importance of Defence becoming a self
critical, self evaluating and self correcting organisation. To do so, the
committee believes that leadership, accountability and the correct alignment of
responsibilities is required. This means that roles and responsibilities should
be clearly defined so that a single point of accountability is attached to
single senior positions, not organisational structures. These roles and
responsibilities should also be complementary.
The headline recommendations deal with much needed organisational change
in Defence in order achieve the correct realignment of responsibilities and functions
of relevant agencies, and provide them with the skills and resources they need
to fulfil their obligations. In this context, the emphasis is on bringing
increased clarity and transparency to responsibility and who owns it—to reduce
the diffusion of responsibility and decision-making. More specifically, the
recommendations are intended to:
- return responsibility to capability managers and make them
accountable for decision-making and performance under their areas of authority;
- make DMO a streamlined and specialist acquisition agency;
inject real contestability into decision-making and guarantee
that the government is provided with independent advice from key agencies—DSTO,
DMO and technical experts; and
- ensure that Defence's focus is on obtaining the right people with
the right skills and experience and matching their skills with the right job:
that Defence also manages its skill base in such a way that agencies complement
their skill requirements and do not compete for skills from the same pool of specialists.
The committee proposes a model that, after second pass decision, allocates
one single point of accountability for every project to the relevant capability
manager, supported by financial delegation and budget control. It reduces CDG’s
role with savings, and limits DMO's functions—thus eliminating much overlap. It
also reinforces the Kinnaird/Mortimer concept for internal independence for the
purposes of genuine contestability, and it reduces the waste of skill through
inappropriate placement, duplication and misalignment of skills. The
committee's proposal also introduces a direct client/provider model without any
intermediaries, with precise accountability. Under this model, the DMO would
become a contract and project management specialist supporting the capability
manager at relevant points in the acquisition and sustainment cycles.
This model would remove the unnecessary layers of current vested
interests and streamline the process through a single point of accountability. In
short, it is a greatly simplified model aided by significant streamlining. It builds
on the strengths of accountability in the services (as identified by the Black
Review) and seeks to harness the learning and potential for alignment across
the three services envisaged with the creation of DAO and DMO.
Strategic Policy Division
The committee recommends that all matters concerning strategic planning,
capability planning, industry policy, costing and all matters for the
coordination of contestability from DMO, DSTO and industry should remain with the
current Strategic Policy Group and CDG in combination.
The committee recommends that Strategic Policy Group and CDG should have
more strategic analytical skills to test rigorously and independently the
capability managers’ development of the Defence White Paper capability
elements, restoring the creative tension but free of competition for skills.
The committee recognises that capability managers have been sidelined
with CDG and DMO assuming key positions during the acquisition phase. To ensure
that they have the power and capacity to discharge their duties, capability
managers require the authority that now resides with the CDG as departmental coordinator
and centre of power. In the committee's view, the priority should be on giving
the capability manager appropriate control over the acquisition ensuring all
the while that the responsibilities of CDG, DMO and the capability managers are
The committee recommends that accountability for all service specific
procurement items should be exclusively transferred with budgets to service
chiefs, who should be responsible for all procurement and sustainment of their
materiel. This transfer of responsibility occurs after proposals have been
thoroughly tested internally and externally and after government decisions are
made at second pass.
The committee recommends that the capability manager should have expanded
responsibility and importantly financial responsibility after second pass.
Under the committee's recommended model, for all acquisition projects, the
capability manager would be the sole client with the contracted suppliers; DMO's
role being limited to tendering, contracting and project management
specialities, strictly according to the terms of the second pass decision. All
specification changes should be monitored by CDG and put to government for
agreement, as currently the practice, with the capability manager to be fully
The committee is of the view that in considering the restructuring of
the organisation, Defence must look closely at the skills required by the
respective agencies and while maintaining strong contestability, ensure that
specialists are located where they are most needed and not unnecessarily
duplicated or spread too thinly throughout the organisation.
Clearly, capability managers need to have the technical experts within
their service able to provide high level specialist advice on a project
proposal from its inception through acquisition, delivery and sustainment. They
should have the responsibility for growing, developing and retaining that
skills base. This is particularly so, given that under the preferred model they
are to be largely responsible for technical input before and after
contract—that is at the heart of the new accountability the committee seeks to
The committee recommends that, after second pass, capability managers have
sole responsibility for acquisition projects, supported by staff seconded
through the DMO, as well as maintaining relationships with contractor and sub
Capability Development Group
The committee recommends that all matters of coordination, overall
budget management monitoring and reporting after second pass should remain in
the current CDG, but without budgetary control.
Defence Materiel Organisation
If capability managers are to be empowered; if they are to exert greater
control over the acquisition of a capability they will use, then DMO's role
must change as well. To be effective, DMO must be responsible for the standards
to be applied to tendering, contracting and project management and have
independent access to the minister.
Responsible for setting the standard for contracting and project
management for the acquisition of the majority of capital equipment, DMO is a
key stakeholder in the capability life cycle. It brings a particular
perspective to capability development and its voice should be included in
advice to CDF and Secretary, and to minister and cabinet. For the purposes of
genuine contestability, organisations such as DMO and DSTO should be truly
independent of Defence with accountability direct to the minister pursuant to
ministerial directive. The Ministerial Directive was a key accountability
document for defence capital projects between 2005 and 2008. It established the
CEO DMO's direct obligations to the minister, his overarching responsibilities
and his management priorities in relation to DMO's business outcomes.
The committee recommends that the minister review, update and reinstate
the Ministerial Directive to CEO DMO. The directive is intended to set
boundaries and expectations and establish clear accountability for achievement
of Defence capital acquisition programs. It should include the requirement that
CEO DMO provides independent advice to the minister in DMO's specialist area of
major capital projects.
The committee recommends that the government should again look carefully
at making DMO a statutorily independent agency, as previously recommended by Kinnaird
and Mortimer, but rejected by Defence and government. The CEO’s salary should
be set by the Remuneration Tribunal and, as stipulated in the previous
recommendation, direct access to the minister should be restored pursuant to a
re-instatement of a ministerial directive which has fallen into disuse. The
intention behind this recommendation is to find a better way to: guarantee
DMO's independence and assist it to provide frank advice to government, have
its functions and responsibilities spelt out in legislation and allow it more
latitude to employ specialist personnel.
Specialist acquisition organisation
For almost a decade, DMO has been actively endeavouring to make itself a
more business-like organisation with the required skills to function as a high
performing acquisition agency. The committee recognises the work the
organisation is doing to achieve that objective but notes that it is falling
short. The committee notes in particular that the skills of uniformed personnel
seconded to DMO may not match the tasks they are required to undertake, which
is further complicated by their short term tenures. The committee recognises
that the DMO needs highly skilled project managers and also tendering and
The committee recommends that
the government ensure that the DMO has the funds, means and government support
necessary to consolidate and build on the efforts already underway to develop
its multidiscipline skills base with the ultimate goal of achieving a
world-class acquisition community.
The committee recommends most strongly that the organisational changes
specified in the recommendations dealing with skills be adopted, and that the
streamlining and consolidation of skills identified be the primary focus and
outcome in securing that change.
Defence Science and Technology
The committee recommends that the minister consider how best to ensure
that DSTO's specialist advice on technical risk associated with Defence's major
capability developments are conveyed to government in a clear and accurate way.
The Ministerial Directive to CEO DMO may serve as a model.
The committee recommends that the Technical Risk Assessments and Technical
Risk Certifications (currently presented to the Defence Capability Committee
and the Defence Capability and Investment Committee) should be a joint activity
overseen by the relevant Service T&E agency head and the Chief Defence
Scientist. In light of past underestimation of technical risk, the intention
would be to review past experiences and current documentation to determine how
risk assessments could be better presented to non-technical experts to minimise
the opportunity for risk assessments to be misinterpreted.
The reporting structure also needs to be transparent such that assessments
cannot be ignored without justification to the key decision-makers (e.g. minister).
In addition to changing the organisational structure, the committee also
recommends action be taken on matters such as the proposed purchase of the 12
new submarines, Air 8000 Ph 2, the 2013 White Paper, the DCP, MAAs and gate
reviews and finally on T&E.
Future submarines SEA 1000—applying
The committee is very concerned about the current unease expressed by a
number of defence analysts regarding decisions already taken on the 12 new
submarines. Recent announcements in relation to studies to consider procurement
options for the future submarines together with studies in relation to an
industry skills plan are encouraging. Nonetheless, early decisions reflect
troubling signs that one of the centrepiece projects listed in the 2009 White
Paper is yet to undergo thorough analysis and consideration.
Because this project is still at an early stage, and based on the RAND
study, the Coles Report, independent defence analysts and the past performance
of major Defence acquisition projects, the committee recommends that government
and Defence start work immediately to:
- ensure that the program is directly managed by Chief of Navy
supported by the ASC and DMO where relevant, the scientific community and the
public—support must be both external to the program and internal within the
navy and submarine community;
- avoid early lock-in through premature weapons systems choices;
ensure that the capability sought is available and minimises
- take drastic action to address the serious skill shortages
identified by RAND before a decision on assembly in Australia is made,
regardless of type and design;
- ensure that the program is open and transparent—full disclosure
throughout the program is necessary to obtain government, industry and public
- involve experienced people in key management positions—this
requires a strategy to grow people so they are experienced in various
disciplines—a top-level strategic lesson must be implemented far in advance of
any specific program; and
listen to technical community concerns about risk—the technical
community, supplemented by outside expertise from industry and allied technology
partners as necessary, should understand the state of technology and the degree
to which a new design extends that technology.
The committee recommends that government and Defence respond publicly to
the committee's criticisms made in this report with respect to lessons not
learnt, and outline the detailed process and all the options on which current
planning on submarines is taking place.
The new White Paper presents an opportunity for the government and Defence
to start to provide assurances that the decisions relating to SEA 1000 are
based in sound, robust and fully considered analysis.
AIR 8000 Ph 2 (Battlefield Airlift—Caribou replacement)
Intended to enhance the ADF's intra-theatre and regional airlift
capability, the Air 8000 Phase 2 project focuses on the provision of an
intra-theatre airlift solution with some inter-theatre application.
According to the Capability Plan, Phase 2 'will provide appropriate training
support, which could include the provision of a Full Flight Simulator'.
The project has been accelerated in order to benefit from the
advantageous pricing through an FMS case. It is assumed by Defence to be an OTS
acquisition and therefore low risk.
The committee's attention is drawn, however, to the incomplete state of the US Air
Force military certification activities. It appears that to date, Defence has
not tasked people qualified and experienced in risk identification to complete
a detailed evaluation of the gaps in capability and certification nor of the
suitability of proposed training simulators to meet training needs. The pattern
appears ominously familiar to the committee.
The committee recommends that the
Chief of Air Force as the relevant capability manager require a report by the
relevant T&E agency against the approved Statement of Operational
Requirement to provide early identification of potential issues that could
delay introduction into service.
The 2013 White Paper—clarity on
The committee's concerns in relation to the accuracy of Defence costings
are reflected throughout this report in terms both of the overall budget and
individual projects. In relation to transparency, the committee emphasises that
greater detail needs to be provided in the Defence White Paper, portfolio
budget statements and Defence Annual Reports. The committee urges the
government to ensure that the 2013 White Paper provides
clarity on future capability including funding commitments and scheduling underpinned
by comprehensive analysis. The primary step toward better alignment between
strategy and capability development would be to ensure that the White
Paper—'the corner stone document'—sets out a realistic and achievable program
for capability development.
The committee recommends that the 2013 White Paper is prepared in such a
way that all procurement proposals are costed and scheduled realistically and
that Defence undertake comprehensive consultation with industry before
decisions on inclusion are made, or alternately, a green paper is issued in
advance for broader and open public consultation.
The committee recommends that, commencing next financial year, Defence
publishes as an addendum to its portfolio budget statements, all the current
financial detail of planned capability from the time of inclusion in the DCP,
right through to contract completion and provision for sustainment, for all
projects over $30 million for total procurement and lifelong sustainment.
Improving gate reviews
The committee has registered its lack of confidence in the effectiveness
of the measures that Defence has taken to improve its procurement
processes—project initiation and review boards, project directives, and gate
review boards. This is not to say that the improvements to these mechanisms
should not go ahead. As noted earlier, the committee is concerned about adding
to an already bureaucratically overburdened process and any such measures to
improve or introduce boards, directives, charters and agreements must always be
guided by the principle of simplification wherever possible. With this
principle in mind, the committee supports for example the strengthened gate
reviews and believes that within the right organisational structure they hold
A number of witnesses strongly supported Defence's revamped gate reviews
which are an improvement on their predecessors especially the inclusion of two
independent experts. The committee, however, does not want to see the
contribution of gate reviews rendered ineffective because of a fundamentally
flawed management structure. The committee underlines the importance of Defence
ensuring that the members of the gate review boards have the relevant skills,
knowledge and competencies to scrutinise the proposals before them effectively.
The committee would like to see the independence of the external members
guaranteed and their ability to provide genuine contestability assured.
In this regard, the committee believes that the gate reviews should be
overseen by a body that can exert its independence and authority to ensure that
gate reviews remain at arm's length from the influence of those with a vested
interest in the project under consideration. The contraventions identified by
ANAO require Defence to look carefully at ways to safeguard the integrity of
these reviews. The committee would also like to see concrete measures taken to
ensure that the implementation of recommendations made by the review boards are
monitored, recorded and reported to the relevant capability manager, CCDG and
The committee notes concern about the gate reviews losing their potency
and simply becoming part of the process if overused. The committee believes an
annual gate review for major projects would add value but recognises that the
format and/or structure may need to be scaled to suit project scope/cost. The
committee recommends that full gate reviews be:
- mandatory for major projects at the following specified
milestones—DCP entry; project initiation and review board consideration; first
pass approval; second pass approval; contract solicitation and contract
- mandatory when a project starts to diverge from original cost or
schedule or when significant changes to scope are proposed.
In light of revelations about breaches of policy such as chairs of
boards having line management responsibility and of misunderstandings stemming
from the documentation provided to the gate review boards, the committee
recommends further that the Independent Project Performance Office (IPPO):
- exert stronger compliance checks to guarantee the independence
and impartiality of the gate review board particularly enforcing the
requirement that the chair of the board must not have line management
responsibility for the project under review; and
- exercise greater scrutiny of the documentation provided to the
review board to ensure that it is relevant and complete including reports on
To ensure that the IPPO has the authority and resources to
discharge its functions, the committee further recommends that Defence consider
carefully whether the functions of the Office should be located in CDG or
With regard to ensuring that the recommendations of the review boards
are implemented, the committee endorses the ANAO's recommendation that 'Defence
ensures that a control mechanism be deployed to monitor the status and
completion of actions recommended by Gate Review Assurance Boards and agreed by
the relevant executive'.
Test and evaluation—building capability
In its report on materiel acquisition and management in Defence, tabled
in March 2003, the committee expressed a lack of confidence in Defence's
'capacity or will to address T&E concerns seriously'. Five years later, in
its 2008 T&E Roadmap, Defence highlighted a raft of shortcomings in
Defence's T&E pointing to a need for greater funding, improved training and
attracting and retaining skilled and experienced personnel. Now, Defence is
still talking about producing a manual—that is about process.
The committee recommends that the government make a long-term commitment
to building technical competence in the ADF by requiring Defence to create the
opportunities for the development of relevant experience.
The committee recommends that capability managers should require their
developmental T&E practitioners to be an equal stakeholder with DSTO in the
pre-first pass risk analysis and specifically to conduct the pre-contract
evaluation so they are aware of risks before committing to the project.
Given that the capability to conduct this T&E and analysis needs to
be extant prior to the commencement of any given project, the committee is
concerned that cost pressures will lead individual services and projects to
degrade this capability over time.
The committee recommends:
- the immediate finalisation of central defence policy on T&E
to be implemented by capability managers in line with the committee’s
recommended shift of full accountability for capability managers for all technical
assessment of capability procurement and sustainment (independently assessed in
conjunction with DSTO);
- full responsibility for the implementation of prescribed T&E
processes be assigned to capability managers for all procurement activity from
inception through to acquisition and sustainment; and
- each capability manager should ensure adequate skilled resources
to oversee all T&E activity in line with central policy, as part of all
acquisitions, including MOTS, as part of the capability managers’ total
responsibility for procurement, but prior to as well as after second pass.
The committee recommends that Defence build on the capability already
extant in aerospace to identify training and experience requirements for
operators and engineers in the land and maritime domains and apply these to ADTEO.
Capability managers will need to invest in a comparable level of training to
enable their personnel to conduct (or at least participate in) developmental
testing. The intention is to provide a base of expertise from which Defence can
draw on as a smart customer during the first pass stage and to assist in the
acceptance testing of capability.
The committee recommends that Defence mandate a default position of
engaging specialist T&E personnel pre-first pass during the project and on
acceptance in order to stay abreast of potential or realised risk and
subsequent management. This requirement to apply also to MOTS/COTS acquisition.
Industry—planning for investment
and early engagement with Defence
Defence's approach to its dealings with industry—planning, acquisition
and sustainment for defence projects—is essential for the successful delivery
of Force 2030. Industry's ability to plan for, and invest in, people and
facilities to deliver future defence projects is significantly dependent on the
information Defence provides about its intentions. The DCP and Defence White
Papers are the main public information tools and key planning documents for
Clearly, from industry's perspective, they fall short in providing the level of
certainty and confidence that industry requires to be an effective partner in
capability development. Industry identified the following problems:
- access to information—the White Paper and CDP deemed to be inadequate
- workflows—feast and famine which severely affects industry's
ability to retain staff—undermines business' confidence and willingness to
invest in future projects;
- slow down rate in approvals—affects both confidence and also industry's
ability to hold on to skilled workers;
- industry not engaged early enough in the capability development
- strained relationship between DMO and industry due to DMO's lack
of business acumen, poor contracting practices;
- contracting arrangements a disincentive for industry to value
- inappropriate risk sharing, fixed price.
The committee recommends that Defence make their DCP a document that
provides industry with greater certainty about its plans and intentions for future
capability development to enable industry to invest with confidence in
capability development. In particular, the committee recommends that the next
- a schedule that provides anticipated timelines for the construction
and delivery of all DCP items, with continuity the key feature;
- a detailed explanation on this acquisition schedule indicating
the reasoning and analysis behind it and how Defence has taken into account
demand flows; and
- reliable cost estimates.
Increased industry involvement earlier in the capability development
process is clearly an important factor in the successful delivery of defence
capability. In this regard, the committee welcomes the reinvigoration of the
Capability Development Advisory Forum and the environmental working groups.
They provide an ideal opportunity to involve senior defence industry
representatives early in the capability development phase without compromising
the integrity of an acquisition process.
The committee recommends that Defence:
- continue to collaborate with industry to reinvigorate the
Capability Development Advisory Forum and the associated environmental working
groups as a means of engaging industry early in the capability development
process. The committee recommends further that Defence ensure that such
engagement with industry is a genuine two-way exchange of ideas and of
- continue to support training programs such as Skilling
Australia's Defence Industry (SADI).
Given the reach back capacity of primes and their ability to tap into
research and development of US and European headquarters, the committee
recommends that industry consultation start at the earliest Defence White Paper
and Defence Capability Plan stage.
Senator Alan Eggleston
Navigation: Previous Page | Contents | Next Page