Chapter 6
Performance benchmarks, accountability and research
Introduction
6.1
This chapter will address terms of reference (g) and (h) and examines the
issues raised in relation to benchmarks, accountability and research.
6.2
In June 2014, DFAT released a new performance framework for the
Australian aid program titled Making Performance Count: enhancing the
accountability and effectiveness of Australian aid. A key principle
underlying this performance framework was 'that funding at all levels of the
aid program will be linked to progress against a rigorous set of targets and
performance benchmarks'. Making Performance Count set out a number of high-level
strategic targets to assess the aid program against key goals and priorities.
These included:
-
Delivering on commitments: From July 2015, progress against
mutual obligations agreed between Australia and its key partner governments and
organisations will form part of program performance assessments;
-
Working with the most effective partners: By July 2015, design and
apply new systems to assess the performance of the aid program's key delivery
partners and ensure stronger links between performance and funding;
-
Ensuring value-for-money: Deliver high standards of
value-for-money in at least 85 per cent of aid investments. Where standards are
not met and improvements are not achieved within a year, investments will be
cancelled; and
-
Combatting corruption: Develop and implement new fraud control
and anti–corruption strategies for all major country and regional programs by
July 2015.[1]
6.3
In the context of PNG, DFAT's aid performance report for PNG in 2014-15
identified a need for 'better defined performance benchmarks for Australia's
contribution to many sectors'. It also noted that the availability of timely
and accurate data sets upon which to base an assessment of progress against
benchmarks 'remains a challenge'.[2]
6.4
The Aid Investment Plan for PNG proposed 'a number of performance
benchmarks be used to assess progress towards the strategic objectives of the
Aid Investment Plan'. Progress against these benchmarks would be reported in
the annual performance report for the PNG aid program. The performance
benchmarks contained in the Aid Investment Plan are narrow and specific
in relation to the outcomes. For example, in 'Outcome 3.2: The quality of key
services in health and HIV/AIDS are improved' the performance benchmark is
'Number of additional births attended by a skilled birth attendant'. The target
for 2015-16 is 9,250.
6.5
The Aid Investment Plan also proposed that Australia and PNG
agree to a set of mutual obligations for the aid program. These mutual
obligations were largely confirmed in the PNG-Australian Aid Partnership
Arrangement 2016-17. In particular, this included a mutual obligation to
continue to implement the Joint Understanding between Australia and Papua
New Guinea on further bilateral cooperation on health, education and law and
order (Joint Understanding). The Joint Understanding included obligations
for PNG to fund 50 per cent of the redevelopment of the Lae Angua Hospital and
funding its ongoing recurrent operational costs' and an undertaking to 'increase
funding to expand and better equip and train the RPNGC'.[3]
Performance benchmarks
6.6
The value of performance benchmarks in the Australian aid program was
repeatedly questioned. Dr Howes from the Development Policy Centre at ANU noted
that this was the fifth effort in two decades to make performance benchmarks
central to the program of Australian aid to PNG. He argued they were of limited
utility. While benchmarks could provide useful information, they 'should not be
expected to improve mutual accountability or influence the size of the aid
program'. He stated:
For these benchmarks to influence aid performance, two
conditions need to be met: the Australian government has to be ready to reduce
or increase its amount of aid in response to performance; and the PNG
government has to be ready to adjust its performance accordingly. There is no
evidence at all that either of these pre-conditions hold.[4]
6.7
A key problem was that if benchmarks are not met in the aid program,
this did not reveal whether the performance was bad or if the targets set were
too high. This risk of potentially punishing good performances in challenging
environments was also highlighted by others. For example, Dr Claxton and Mr
Jennings from ASPI stated:
But while positive results can be usefully incentivised, we'd
caution that where key national interests are at stake, benchmarks should be
strategic ones (such as those measured in the latest PNG Aid Program
Performance Report) and assessed strategically. It could be strongly against
our interest to penalise poor performance in important projects on tactical
rather than high level criteria. And where really crucial interests are at stake,
it could be more appropriate to redouble than withdraw support when we’re
expectations aren't being met.[5]
6.8
This issue was seen as especially relevant to PNG, where the conditions
made it difficult to undertake development assistance projects and to demonstrate
that these projects have had an impact. World Vision advised that the 'Australian
Government must recognise the complexities present in PNG as well as their
impact on the visibility of progress'.[6]
The Lowy Institute cautioned that 'Australian aid cannot address all
development challenges in [PNG]'. It encouraged a realistic approach:
In the delivery of a very large aid program there are likely
to be some poor decisions made or some failures of implementation. But this
does not mean that Australian aid has not been useful. Australian expenditure
in the health sector in Papua New Guinea has saved lives.[7]
6.9
The OECD DAC observed that 'accountability for results needs to be
addressed at two levels: accountability to the partner country and
accountability at home to show that public funds achieve results'. It stated:
Australia, like other DAC members, needs to get the right
balance in its performance system so that results information is (i) useful for
improving the quality and impact of development co-operation and (ii) it
enables the donor to communicate credibly about the long-term development
results Australian aid is achieving.[8]
6.10
A variety of views were expressed in relation to how benchmarks could be
framed. For example, ChildFund Australia recommended that the Australian aid
program 'apply benchmarks that do not rely on generalised economic growth, but
inclusive growth... benchmarks which ensure that our aid targets the poorest 40
per cent of people in middle and low income countries'.[9]
Jubilee Australia argued that the '[t]argets established to assess aid outcomes
should be developed against locally-derived criteria documenting the
aspirations of local communities in PNG'. While World Vision considered that
there should be a focus on outcomes rather than outputs:
[I]n addition to an outcomes-focus, there must also be an
understanding of the long-term nature of sustainable outcomes, particularly in
relation to deeply embedded social and cultural issues such as gender norms
which underpin persistent inequalities and acts of violence against women and
girls. Targets pertaining to social change initiatives must retain a level of
flexibility that reflects the long-term yet critical and foundational nature of
this work. [10]
6.11
The National Research Institute argued for benchmarks more closely
aligned with the priorities of the PNG Government:
For Australian Aid to be effective in its delivery, it must
develop internal systems for ranking initiatives according to the progress that
would be made against the targets laid down in the PNGDSP and MTDP. In this
way, Australian Aid would transparently be directing funding to the development
and wellbeing of PNG in accordance with the priorities of [the PNG Government].
Such an approach for Australian Aid would be a radical departure from the
existing consultancy based support for public administration.[11]
Monitoring and evaluation
6.12
While performance benchmarks were considered to be limited in their
usefulness, the monitoring and evaluation of projects and data collection
activities were highlighted as valuable in effectively directing resources. ACFID
noted that the 2014 report on Australian aid had found that 'while monitoring
and evaluation across programs in the Pacific was below the global average, the
situation in PNG was far worse where less than 50 per cent of investments had
satisfactory monitoring and evaluation arrangements'. It recommended a stronger
focus on 'capability building in monitoring, evaluation and learning across the
PNG program'.[12]
Similarly, the Burnet Institute perceived a need for dedicated resources to
strengthen the capacity for monitoring and evaluation in PNG. In particular, it
considered that all monitoring and evaluation frameworks for Australian funded
activities in PNG include equity indicators.[13]
6.13
The practical value of monitoring and evaluation activities were
outlined by several organisations which deliver aid to PNG. For example, ACIAR
highlighted its use of impact assessments and adoption studies to ensure it 'actively
measures its effectiveness, learns from experience, and adjusts or cancels
projects that are not achieving results':
ACIAR's impact assessment program is devoted to provide
independent assessment of the performance of ACIAR's investments...ACIAR has made
a practice of revisiting a sample of past projects some time after their
completion, and now has a series of adoption studies that spans the past
decade. ACIAR commissions specialists to undertake assessments 3–4 years after
a large project is completed to determine the level of uptake of the findings
and gauge the extent of the project's legacy.[14]
6.14
Vision2020 illustrated the issues that uncertainty in benchmarks and
data collection could create. It outlined that there was a lack of data consistency
between members of the PNG National Prevention of Blindness Committee including
continuing disagreement amongst NGOs regarding definitions for collection
criteria (for example, the age range for who constitutes a child is disputed).
It stated:
Development of a consistent and user friendly data collection
and management system would make considerable progress towards improving
accountability of eye health and vision care in PNG. Support for a simple
monitoring and evaluation (M&E) system that is driven by and universal to
all eye health stakeholders (including public, private sector, civil society
and churches) would result in increased accountability across the sector and
the ability of the sector to respond to evidence based demand.[15]
Mutual accountability
6.15
While mutual accountability or mutual obligations are a key part of the PNG-Australia
Aid Partnership Arrangement and an increased focus of the Australian
Government, this aspect of the aid relationship with PNG did not receive
significant attention during the inquiry. One organisation which did provide commentary
was Save the Children. It argued that:
[M]utual accountability should focus on delivering
improvements to service delivery. That is, using our aid investment to help
create the right incentives and other conditions to ensure the PNG Government
uses its own resources more effectively and efficiently.[16]
6.16
The challenges of setting mutual obligations were illustrated during the
inquiry. Under the Joint Understanding Australia would provide $420 million
of assistance including funding to support the master plan and scope of works
for the redevelopment of the Lae ANGAU Hospital, funding 50 per cent of the capital
costs associated with redevelopment and contributing to the costs of senior
management personnel. In return, the PNG Government would meet 'its commitment
to fund 50 per cent of the redevelopment of the Lae ANGAU Hospital and funding
its ongoing recurrent operational costs'. However, there does not appear to be
any subsequent announcement regarding PNG Government funding for the project
and media reports have suggested that the PNG Government was pressing Australia
to commence further work on the project before its financing was confirmed.[17]
Fraud and corruption
6.17
While ineffective governance was highlighted as a major impediment to
development in PNG, fraud and corruption was also raised as a related obstacle.
PNG ranked 139th of 168 countries on Transparency International's
Corruption Perceptions Index 2015.[18]
Transparency International PNG has observed that:
PNG has some of the best rules and laws in the world, but the
biggest problem continues to be the lack of enforcement of the law. There is
little or no accountability for those who fail to follow the rule of law in
dealing with state assets and decisions. The legal loopholes and an apparent
lack of political will facilitate domestic and cross-border corruption with
offenders enjoying scandalous levels of impunity.[19]
6.18
Many submissions argued that the Australian Government needed to do more
to address the incidence of fraud and corruption in PNG. The Reef and
Rainforest Research Centre stated that a '[s]trong emphasis on investigating
and addressing corruption in the use of Australia Aid funds is essential to
improving delivery'. In its experience from operating in Western Province was
that corruption at a political and bureaucratic level was 'the single largest
obstacle to socioeconomic development'. The International State Crime
Initiative also argued that the provision of aid to PNG must 'take into account
the profound impact state/corporate criminality has on security, economic
participation, human rights and resource management'.[20]
6.19
The joint submission from PNG Attitude urged that 'Australia should
not shirk its responsibility to harness its aid program to PNG to honest,
efficient and accountable governance':
Tackling corruption is urgent and should not be ignored by
the Australian Government, no matter what the diplomatic intricacies may be.
Australia should make the establishment of an independent and effective
Independent Commission Against Corruption or similar entity a pre-condition for
granting aid.[21]
6.20
Worryingly, there were indications that misappropriated Australian aid
money was flowing back into Australia. For example, the Pacific Future
Foundation outlined its concerns that 'current foreign aid given to PNG has
assisted in contributing to a system with endemic corruption':
During a number of conversations with ex-pat Australians it
became apparent that it is general knowledge amongst this community that
Australian Aid money does not reach its intended targets and when it does only [paltry]
amounts reach the people in need. The consensus amongst the people we met was
that at least half the aid funding returns to Australia via Cairns where senior
PNG government officials now own a significant number of residential
properties. They have observed that aid money is accessed to spend on travel to
and from Cairns for the lifestyle offered away from the rigours of Port Moresby.[22]
6.21
Save the Children pointed to corruption as a key reason economic growth
has not translated into large-scale poverty reduction in PNG. It stated that
while stemming corruption in the aid program is important, 'the big ticket item
is combatting corruption in PNG's broader public and private financial flows so
that resources are available for public services'. Save the Children
recommended the Australian Government '[a]dopt and publically report measures
to tackle the benefits of PNG corruption from being exploited within Australia
including use of financial and diplomatic instruments and support for
Australian anti-corruption agencies to investigate PNG investments'.[23]
6.22
The PNG-Australia Aid Arrangement 2016-17 reaffirmed that both
governments agreed to 'maintain their zero tolerance commitment to fraud in
Australia's aid program'.[24]
This commitment was illustrated during an incident in 2013, when Australia ceased
funding the procurement and distribution of medical supplies in PNG resulting
from a compromised tender process.[25]
Mr Mat Kimberley from DFAT told the committee the Australian Government took
the decision because of a 'questionable' procurement process.[26]
6.23
DFAT noted:
Australia has also prioritised programs that aim to combat
corruption in PNG, including by supporting PNG's investigation and prosecution
capacities. This is assisted through the placement of SGP advisers in the PNG
Office of the Public Prosecutor (OPP), and AFP advisory support to the RPNGC
Fraud and Anti-Corruption Directorate.
We are supporting PNG's efforts to establish an Independent
Commission Against Corruption and to improve its compliance with international
anti-money-laundering and counter-terrorism financing standards.[27]
6.24
In particular, Australian Attorney-General's Department (AGD)/Australian
Transaction Reports and Analysis Centre (AUSTRAC) Combating Corruption project assists
Papua New Guinea (PNG) in preventing and combating corruption by strengthening
its financial system against money laundering, corrupt activity and terrorism
financing.
Research and learning from aid programs overseas
6.25
There was support for 'the general principle that Australia should be
constantly seeking to adopt methodologies and techniques of aid delivery that
have been shown to be efficacious in other jurisdictions'.[28]
However, many submitters argued that the situation in PNG was so different that
lessons from successful aid programs in other countries would not have
significant value. For example, Ms Crawford from IWDA described the
conditions in PNG as so unique that she counselled against the concept that
Australia could 'import specific learnings from elsewhere'.[29]
6.26
World Vision gave examples of successful programs in other countries
which could be applicable to PNG. These included working with communities in
Mozambique to define optimal governance structures and approaches to translate
mining benefits to community driven outcomes and success of the Timor Leste National
Malaria Control Programme. However, it also cautioned:
PNG presents significantly different development challenges
and is unlike any other setting. Any successes or lessons derived from the
delivery of aid programs elsewhere will therefore require significant
contextualisation for the Papua New Guinean setting, including the
participation of communities to ensure the applicability and appropriateness of
new initiatives.[30]
6.27
A number of other potentially applicable programs were suggested. Family
Planning NSW highlighted two programs developed in other countries which could
be implemented in PNG.
PNG has unacceptably high rates of cervical cancer incidence
and mortality. Cervical cancer screening saves women’s lives, however it is not
routinely available in PNG. We have piloted and developed a successful and sustainable,
low resource cervical cancer screening and treatment program in Fiji using VIA
(visual inspection with acetic acid and cryotherapy) that could save the lives
of many women in PNG, if the program and capacity building training in the
method were further implemented there...Similarly, we have implemented an access
to contraception program in Vanuatu that could lead to reduced teenage and
unintended pregnancies in PNG, if also implemented there.[31]
6.28
The Burnet Institute recommended that 'DFAT should undertake a mapping
of successful health care initiatives in low and middle-income countries in the
Asia Pacific region to identify opportunities to share experiences with
government and civil society health officials in PNG. In particular, it
highlighted there were potential lessons to be learned from health development
progress in the northern provinces of Laos and Nepal. It was also recommended
that 'DFAT should look to develop a mechanism for aggregating and sharing
lessons across sectors and from prior aid programs in PNG'.[32]
6.29
Areas of future research were also identified. For example, Coffey
highlighted that in the international development sector, discussions were
underway regarding how mobile phones can provide enhanced and beneficial
services to people living in isolated, poor and disadvantaged communities. It
argued it would be important for the Australian Government to continue to
support ongoing research to help aid practitioners understand the opportunities
and also the limitations of incorporating portable, digital technologies into
program design and delivery.[33]
6.30
ACIAR noted that its support to PNG is 'both bilateral (between
Australian and PNG research and policy institutions) and regional (between
Australia, PNG and other countries, generally other Pacific island countries)':
Outcomes in PNG and other Pacific island countries are
enhanced by the sharing of experiences from successful projects through several
mechanisms, including cross-program teams, thematic workshops, technical
reports and publications, impact assessments, and regional engagements with [the
Secretariat of the Pacific Community].[34]
6.31
Dr Austin from ACIAR outlined that the agency had supported more than
180 research projects in PNG over the last three decades, including 37 active
projects.[35]
6.32
Ensuring research findings were taken into account in subsequent policy
making was also highlighted. An example provided by the CSIRO illustrated this
issue. Between 2011 and 2014, the Australian Fisheries Management Authority and
the PNG National Fisheries Authority funded CSIRO to investigate the status of
small-scale fisheries, livelihoods and food security in the Papua New Guinea
villages bordering the Torres Strait of Australia. One of the findings of the
research was that poaching pressure in Australian waters will increase as
Treaty villages' resources are depleted. The CSIRO noted:
The results triggered a one-off PNG Treaty Villages
Sustainable Development Workshop at the October 2014 Treaty meetings to discuss
the declining situation and investigate remedies. This involved Australian and
PNG government and community stakeholders, and was convened by CSIRO, DFAT and
the PNG National Fisheries Authority. A good governance framework provided by
the Treaty meeting cycle helped the integration of the research results into
policy.[36]
6.33
Mr Schaefer from Save the Children stated:
[B]etter data is key to improving the supply of services and
the demand for them. Better data is key to the measurement of the effectiveness
of aid programs. We encourage the Australian government and the PNG government
to build datasets in Papua New Guinea that can lead to improved service
delivery.[37]
6.34
DFAT stated that it had many systems and processes 'to ensure important
lessons from other countries are captured and integrated into the aid program
in PNG'. In particular, the Pacific Division contains the Pacific Analytical
and Effectiveness Branch, which includes the Pacific Strategy and Performance,
Quality and Risk Sections. It outlined:
Through these teams, data and lessons learned from regional
programs and Pacific bilateral programs are captured and shared within the
Division...DFAT's organisational structure also includes thematic teams that work
across major sectors, and act as a key conduit for the exchange of lessons
across Australia’s various aid investments within DFAT and for implementing
partners.[38]
Committee view and recommendations
6.35
The committee shares the view that performance benchmarks have limited
utility in the context of the PNG aid program. In situations where the
achievement of development outcomes is expected to take decades, annual
performance benchmarks are unlikely to be an effective mechanism to
significantly change behaviour or improve program performance. The benchmarks
contained in the Aid Performance Plan are narrow, however they provide
detailed targets and indicators to track progress in those specific areas
listed. In the view of the committee, this appears to be a realistic and
pragmatic approach.
6.36
The committee urges the Australian Government to continue its support to
PNG, through the Attorney-General's Department, AUSTRAC and the Australian
Federal Police, to combat fraud, corruption and money laundering. These
agencies should also focus attention on corrupt activities in PNG with relevance
to Australia. Last year the risk of money laundering from PNG to Australia was
again highlighted in media reports.[39]
This conduct is a critical reputational risk to Australia's aid program to PNG.
Public support for the Australian aid program to PNG could be damaged if aid funding
is perceived as being misappropriated. Corrupt conduct which involves the
transfer of funds to Australia also reduces the capacity of the PNG Government
to deliver services to its people.
Recommendation 17
6.37
The committee recommends the Australian Government target illegal
activities undertaken in Australia which are linked to corruption in Papua New
Guinea.
6.38
The unique conditions and circumstances in PNG means that caution must
be applied when examining the suitability of transferring aid programs used in
other countries. For the committee, this highlights the need for better
research into development issues in PNG. A better understanding of what works
and what doesn't in PNG will assist the Australian aid program. To some extent
this is already occurring. The committee has been impressed by the insights
provided by the research programs in PNG undertaken by the CSIRO and ACIAR and
the analysis provided by the PNG National Research Institute and the Development
Policy Centre through the Promoting Effective Public Expenditure project. The
key challenge appears to be in translating the findings of this PNG-specific development
research into the reform of the Australian aid program to PNG.
6.39
The committee notes that PNG is listed as 'considering partnership' with
the 'Better data for health partnership' project of DFAT's innovationXchange
and Bloomberg Philanthropies. The committee hopes this occurs as the PNG health
system and the Australia aid program would benefit from one of the key aims for
the project 'to strategically use public health data to inform policy
priorities'.[40]
Recommendation 18
6.40
The committee recommends that the Australian Government:
-
continue to support research activities which promote the effectiveness
of the aid program to Papua New Guinea; and
-
ensure that this research is considered in decisions made
regarding the aid program to Papua New Guinea.
Senator
Alex Gallacher
Chair
Navigation: Previous Page | Contents | Next Page