5.1
Submissions considered the means by which Australian official development assistance might be evaluated. The Department of Foreign Affairs and Trade (DFAT) outlined that when measuring success, it looks at the ‘long-term impacts that might be contributing to that transformational change,’ as well as ‘outcomes and inputs.’
5.2
DFAT’s Aid Programming Guide suggests that ‘outcomes should define:’
an ‘end state’ when the outcome has been achieved;
who or what is expected to change; the type of change expected to occur: knowledge (awareness of new ideas, techniques or strategies);
action (behaviour change based upon new information/ideas); or
condition (organisational or societal conditions changes due to the stakeholder’s actions); and
the time by which the change is expected to occur.
5.3
It was recognised that outcomes for women and girls in the region requires long-term strategies. The Pacific Women program, in particular, was discussed by inquiry participants as an example of long-term development assistance.
5.4
DFAT provides funding through a variety of programs, including: those associated with the Pacific Step-up, such as the Australian Infrastructure Financing Facility for the Pacific; the Australian NGO Cooperation Program; Pacific Women; and bilateral aid.
Evaluating effectiveness of development assistance
5.5
Changes have been made to the DFAT’s performance assessment framework since the Committee’s 2015 report, which examined DFAT’s then-recently revised performance framework, Making Performance Count: enhancing the accountability and effectiveness of Australian aid.
5.6
DFAT stated in its submission to this inquiry that ‘the effectiveness of Australia’s overseas development assistance is regularly evaluated in line with DFAT’s publicly available monitoring and evaluation standards.’ Evaluation is defined by DFAT as:
… the systematic and objective assessment of an ongoing or completed investment, program or policy. It is an in-depth process which takes place on a periodic basis. Evaluation aims to provide credible evidence which can inform major program management and policy decisions and highlight important development lessons.
Evaluation is distinct from, but related to, monitoring. Monitoring is a continuous process which examines whether an investment, policy or program is on track to achieve its intended results.
5.7
The performance assessment process was revised as part of DFAT’s COVID-19 response, Partnerships for Recovery, in May 2020. Continuing from consultations in 2019 on a ‘new international development policy’, this includes performance indicators relating to violence against women and girls, education enrolment rates, women’s entrepreneurship, and percentage of investments effectively addressing gender issues.
5.8
In November 2020, DFAT published good practice guidance on gender equality in monitoring and evaluation and reporting to support the performance measures described in Partnerships for Recovery.
5.9
DFAT described strategic evaluations on its website as ‘broad assessments of Australian aid that focus on policy directions or specific development themes.’ This is differentiated by DFAT from investment- or program-level evaluations which ask whether a specific program has been successful.
5.10
The strategic evaluation work of DFAT’s Office of Development Effectiveness (ODE) was cited by a range of inquiry participants, indicating a good awareness of the insights provided by the ODE. In relation to women and girls in the Pacific, in recent years the ODE has published two substantial reports:
Ending Violence Against Women and Girls: Evaluating a decade of Australia’s development assistance (2019)
Pacific Women Shaping Pacific Development Six-Year Evaluation Report (2020)
5.11
The ODE commissioned its 2019 evaluation ‘to assess the progress made since its 2008 evaluation, Violence Against Women in Melanesia and East Timor: Building on Global and Regional Promising Approaches.’ The ODE’s evaluation work included identifying best practice examples. Femili PNG stated it had been identified ‘as an example of best practice’ in regard to ‘the importance of supporting and strengthening case management services’.
5.12
CBM Australia raised concerns that a lack of baseline data prior to the implementation of the Pacific Step-up has made it difficult to measure the effectiveness of specific initiatives:
Insufficient reporting on the Australian aid program’s efforts to engage with [disabled people’s organisations] on gender equality issues or to advance the human rights of women and girls with disabilities more broadly has led to difficulty assessing recent effectiveness. Without a baseline prior to the implementation of the Pacific Step-up, observers can note broad outcomes but are unable to determine the extent to which specific initiatives under the banner of the Pacific Step-up have advanced human rights in the region.
5.13
CBM Australia also commented that changes in DFAT’s performance reporting had led to a decrease in detail:
Prior to the Step-up, DFAT’s Aid Quality Checks (AQCs) provided some evidence of impactful engagement against two primary metrics: investments identifying and addressing barriers to inclusion and opportunities for participation for people with disabilities, and investments involving [disabled people’s organisations] in planning, implementation, monitoring and evaluation. AQC data following the launch of the Pacific Step-up in late 2018, however, demonstrates ‘business as usual’ for people with disabilities, and does not provide information on disability inclusion within gender-specific aid investments.
5.14
DFAT’s Aid Program Performance Report 2018-19 for the Pacific Regional Program stated ‘in 2018-19, the Aid Quality Check (AQC) process has been revised to focus on fewer quality criteria (effectiveness, efficiency, gender equality).’ Further changes to the AQC process had been made by May 2020, with Aid Quality Checks being replaced by Investment Monitoring Reports.
5.15
Oaktree similarly recommended that evaluations of gender equality initiatives ‘must evolve to collect both age and gender disaggregated data’ to better equip the Australian Government to ‘track the progress towards securing the human rights of women and girls in the Pacific and achieving gender equality.’
5.16
The Fiji Women’s Fund (FWF), and Urgent Action Fund Asia & the Pacific (UAF A&P) stated that a clear picture of DFAT’s funding activity was not publicly available:
It is challenging for Pacific Women’s Organisations (PWOs) and Disabled People’s Organisations (DPOs) in the Pacific to obtain data that presents a picture of total funding for gender equality and women’s empowerment activities, including funds disbursed to women’s organisations and DPOs in the Pacific region in the previous reporting year, in a readily accessible format.
5.17
The FWF and UAF A&P highlighted a disparity between funding commitments and funding disbursements, and advocated that data sets ‘currently sent by donors each year to the OECD [Development Assistance Committee (DAC)]’ be published, including:
Total gross funding disbursements directed to the Pacific region that target gender equality and women’s empowerment as a significant or principal objective (using the OECD DAC criteria)
Total gross funding disbursements that reach PWOs
Total gross funding disbursements that reach Pacific DPOs
The proportion of funds earmarked for women’s organisations and DPOs that are being disbursed to: (i) INGOs, international organisations and managing contractors and (ii) PWOs and DPOs, as well as the names of these organisations.
5.18
The Pasifika Women’s Alliance emphasised greater involvement of women when evaluating the success of development assistance programs as ‘it is preferred that Pacific women are included in the delivery and evaluation of programs to provide context, relatability, and a stronger impact in their communities.’ CBM Australia advocated for the inclusion and consultation of women and girls with disabilities ‘in the program cycle from design to implementation to evaluation.’
5.19
A ‘severe gender imbalance’ in Australia’s foreign policy sector was highlighted by YWCA Australia. The YWCA elaborated that this could be impacting evaluations of progress towards achieving gender equality:
When policy is not inclusive of women’s input and does not seek to evaluate its effectiveness through a gender lens, it becomes an additional and unnecessary barrier to the achievement of gender equality, not only in Australia but in regions such as the Pacific.
5.20
Melbourne Children’s Global Health highlighted that those delivering programs were well-placed to obtain data for robust monitoring and evaluation:
Non-government organisations also have a critical role in improving the evidence-base for interventions to prevent domestic, family and sexual violence. Non-government organisations can incorporate high-quality evaluation of their programs, using robust methodologies that combine both quantitative and qualitative methods to develop a better understanding of the effectiveness of their programs.
5.21
The Australian National University (ANU) Department of Pacific Affairs (DPA) stated that evaluating the progress made for women and girls requires more than program-based evaluations:
The evidence of effective strategies, and of the implementation of reforms in the law and justice sector, remains thin and centred on program evaluations. To ascertain whether the rights of women and girls are being upheld, in multiple contexts, requires extensive collaborative and ethical research with service providers and advocacy networks.
Long-term perspectives
5.22
Submissions outlined that long-term strategies are required for Australia’s official development assistance to be effective in advancing the human rights of Pacific island women and girls.
5.23
The ANU Development Policy Centre and Femili PNG emphasised that gendered violence in the Pacific islands was a long-term problem due to its enduring repercussions:
Family and sexual violence must be recognised as a long-term human rights problem that disproportionately affects women and girls, has multigenerational impacts, and is a constraint on development that requires sustained and serious engagement and investment.
5.24
DFAT recognised that ‘changing social and cultural norms to improve gender equality is difficult and can require decades to see results’. DFAT stated that it had ‘expanded long term funding to key civil society organisations’ and emphasised that:
… Australia has proven it has a long-term commitment to the Pacific region to address gender equality and support immediate service delivery. We recognise that activities and projects are foundational and critical for providing services, training, and empowering women. However, our programs seek to trigger longer term cultural and institutional change through supporting local leadership.
5.25
A distinction between immediate relief and long-term approaches was identified. The Skoll Centre for Social Entrepreneurship and others emphasised that ‘immediate relief measures are necessary to alleviate hardships and protect survivors, the approach of changing mind-sets appears to be more sustainable on a long run.’ The International Women’s Development Agency (IWDA) stated:
While protection is an important first step, it addresses the symptoms of gender equality rather than the causes. It is therefore unlikely to create meaningful and sustainable change to the status of women and girls alone. As one of the Pacific’s strongest allies, Australia’s COVID-19 Response should therefore look beyond protection towards expanding transformational interventions, by investing in policies and programs that challenge the power relations, systems and norms that underpin gender inequality.
5.26
Long-term thinking was described as an iterative process. Cardno stated that ‘successive programs in the Pacific, whether they be gender-focussed or take a gender mainstreamed approach, [should] leverage the gains and learnings, as well as the relationships, of earlier programs.’
5.27
The Australian Council for International Development similarly stressed that learning from current programs is a foundational step in long-term planning:
The Government should ensure that any new investments build on the successes and lessons learned through Pacific Women, as part of a commitment to long-term thinking, building on a program that continues to demonstrate success.
5.28
Two to three year funding cycles were described as inadequate. The IWDA stressed the ‘importance of long-term, continuous funding’, highlighting that short funding cycles restrict the time available for program delivery:
What we are dealing with is an intergenerational issue around gender equality, and we can’t effectively respond to that when we’re working on two- and three-year funding cycles. Often it’s much shorter—at the moment, it’s sort of 18 months or six months. You can’t get into a program; you start activities and then you need to report on it again. It doesn’t give us any basis to be able to address these fundamental issues.
5.29
The ANU Development Policy Centre stated ‘most of our funding comes in for one or two or three years. The Australian government has been very good in giving us three-or five-year funding. Of course it can be repeated, but we think the expectation needs to be that the funding will be provided over a long period.’
5.30
An increased disbursement of funds to civil society organisations from governments in the Pacific was not expected by the ANU Development Policy Centre in the near-term:
The Australian government does prioritise these issues above the level they are prioritised within the government of PNG and of course has a lot more resources, and the problem is going to be there for a long time, so there needs to be long-term security of funding.
Pacific Women Shaping Pacific Development (Pacific Women)
5.31
The Pacific Women Shaping Pacific Development Program, commonly referred to as Pacific Women, was established in 2012. At the time of the Committee’s 2015 report, it was considered that it was ‘too early to assess’ whether Pacific Women was achieving results.
5.32
Pacific Women was outlined by DFAT to have ‘expensed $218.68 million between 2012–2019’ on the following areas of focus:
$18,116,899 (8 per cent) on Leadership and Decision Making.
$49,514,935 (23 per cent) on Economic Empowerment.
$96,184,154 (44 per cent) on Ending Violence against Women.
$32,147,464 (15 per cent) on Enhancing Agency.
$18,578,937 (8 per cent) on administration and program support.
$4,137,448 (2 per cent) on monitoring, evaluation and design.
5.33
The We Rise Coalition recognised the Pacific Women program for ‘assisting thousands of women to step into leadership roles, participate in financial services and training, and access crisis support services.’
5.34
An initiative of the Pacific Women program was recognised by the Law Council of Australia as being especially impactful on women’s economic participation:
… the Markets for Change program, funded by the Australian Government through Pacific Women, and implemented by UN Women, works to ensure that marketplaces in Fiji, the Solomon Islands and Vanuatu are ‘safe, inclusive and non-discriminatory, promoting gender equality and women’s empowerment.’
5.35
Cardno highlighted the success of locally led efforts funded under the Pacific Women program. Cardno provided the example of Balance of Power, a social norm change initiative, stating that it had been ‘accepted as a local initiative’:
Balance of Power is seen and accepted as a local initiative, driven by well-known and respected individuals from the local community. It is not perceived as a donor-driven, foreign ‘intervention’. This is particularly important in the case of gender programming, given the complexity of promoting norm change on issues that are so deeply, culturally entrenched.
5.36
The ANU DPA recognised the Balance of Power program for being flexible towards changing priorities due to the coronavirus pandemic:
Programs that are heavily pre-determined in terms of their activities and deliverables have been the hardest to reshape in the aftermath of the 2020 pandemic. … The program is intentionally adaptive, and leaves decisions about activities and partners to the core leadership team who have an intuitive understanding of their local contexts, including power bases and opportunities to build coalitions.
5.37
The Pacific Women program has a range of priorities, and it was highlighted that individual initiatives may still experience funding constraints. Oaktree stated that youth programs under the Pacific Girl initiative ‘relies on a relatively tiny budget, limiting its potential to facilitate progress toward gender equality’.
5.38
Caritas Oceania drew attention to the limited 10-year nature of Pacific Women, stating that ‘funding … is set to expire in 2021-2022.’ Caritas Oceania advocated that the program be renewed.
Pacific Women Lead
5.39
In June 2021, DFAT advised that it was in the process of co-designing its Pacific Women Lead program as a successor to Pacific Women Shaping Pacific Development. DFAT stated:
Pacific Women Lead sees Australia increase its financial commitment to $170 million over five years, which is in addition to our bilateral gender support delivered through our bilateral development programs. The new program shifts program ownership to Pacific organisations, including the Pacific community, the SPC and women’s organisations in the Pacific. Pacific Women Lead builds on the achievement response to the lessons learned from the Pacific Women Shaping Pacific Development program, and will be flexible enough to respond to the emerging needs of women and girls, including through the post COVID-19 pandemic period as well as climate change.
5.40
The co-design process for the Pacific Women Lead program was announced on 24 November 2020. The material published by DFAT states that ‘Pacific Women Lead has an initial timeframe of 5 years with the potential for an extension to 8 years’.
5.41
DFAT stated that ‘consultations and co-design with the region’ had identified areas ‘where we can do things differently.’ DFAT highlighted that Pacific Women Lead will focus further on programs lead by women in the Pacific directly:
We’ve reached the stage now, with our support to the region … where there will be a shift so that Pacific women themselves and Pacific organisations will lead the programs in ways that they haven’t before. … In particular, the Pacific community and Pacific women’s organisations, which … through DFAT’s support over the years have grown and strengthened themselves, will lead a lot of the elements of the program.
5.42
The IWDA welcomed the Pacific Women Lead program, but stated that:
However, this commitment is only for regional activities. It’s imperative, if Australia is to continue the hard won gains under the current gender programming, that bilateral aid programs also have a target for their gender expenditure. Currently only 41 per cent of Australia’s ODA has a principal or significant focus on gender equality.
5.43
DFAT stated that Pacific Women Lead has been designed so that it ‘is easy for donors’, such as the European Union, ‘that don’t have a large presence in the Pacific to provide funding’.
Concluding comment
5.44
The Committee heard that to be effective in solving long-term challenges, long-term funding is required. This is not a new problem for official development assistance. The Committee heard that some thought has been given to this issue, with some organisations receiving three-year funding terms. The Committee heard that three-year or less funding cycles can hamper stable program delivery, and that five-year funding cycles would be preferred.
5.45
The Committee notes the success of the Pacific Women Shaping Pacific Development as a ten-year program in its approach to long-term development aid planning. The Committee heard that through this program DFAT has recognised and demonstrated a long-term commitment to advancing the human rights of women and girls in the region.
5.46
The Committee understands that the Pacific Women Lead program is currently a five-year commitment of funding, and considers this inconsistent with DFAT’s previous success in managing the long-term injection of funding through the Pacific Women Shaping Pacific Development program.
5.47
The Committee understands that the Pacific Women Lead may, however, be extended for a longer period. The Committee also notes that the Pacific Women Lead program aims to building on the successes of the Pacific Women Shaping Pacific Development program, based on DFAT’s intention to take the support for locally led initiatives further.
5.48
To determine whether outcomes are being achieved for women and girls in the region, the Committee heard that a single evaluation methodology is not sufficient. Investment or program-based evaluations are important, but are necessarily limited by scope in the conclusions they can draw.
5.49
The Committee considers that methodologies that evaluate a wide range of programs across a specific theme to be essential to directly evaluating whether Australia’s official development assistance is leading to progress for women and girls. This is particularly so as Australian Government funding for these initiatives is not neatly encapsulated in a single program, but across a range of programs.
5.50
Based on the evidence received to this inquiry, the Committee considers that the Office for Development Effectiveness was a model for evaluation that was well-received. The JSCFADT acknowledges DFAT’s evidence from its COVID-19 inquiry that the Office for Development Effectiveness was abolished in late 2020, and evaluation functions moved into DFAT’s Office of the Chief Economist.
5.51
The Committee heard that an overall picture of DFAT aid funding is difficult to ascertain on a year-to-year basis, and that external scrutiny and evaluation would be assisted by regular publication of funding disbursements by the Australian Government.
5.52
The Committee recommends that the Australian Government consider long-term funding cycles for official development assistance to be at least five-years.
5.53
The Committee recommends the Australian Government extend its current five-year commitment to the Pacific Women Lead program to allow for funding cycles of this duration to be provided and to include a target for its gender expenditure.
5.54
The Committee recommends that the Australian Government commit to continuing to undertake and publish strategic, thematic evaluations focusing on substantive human rights issues affecting women and girls.
5.55
This should allow evaluations of the full range of official development assistance, and extend evaluations beyond those focused only at the investment or program-level.
5.56
The Committee recommends that the Australian Government improve monitoring, evaluation and reporting of development assistance programs to ensure transparency and easy access to data, especially including reporting on funding for gender equality and women’s empowerment activities in the Pacific.
5.57
The Committee recommends that the Australian Government, on an annual basis, publish data on the disbursements made to organisations for the purpose of gender equality initiatives. This should list the organisations receiving disbursements.
Senator the Hon David Fawcett
Chair
Joint Standing Committee on Foreign Affairs, Defence and Trade
24 November 2021
The Hon Kevin Andrews MP
Chair
Human Rights Sub-Committee
24 November 2021