3.1
This chapter considers the performance of the Australian Criminal Intelligence Commission (ACIC) against its outcome in the Attorney-General Portfolio Budget Statements 2017–18 (PBS), and its key performance criteria as outlined in the ACIC Corporate Plan 2017–18 to 2020–21 and Strategic Plan 2016–21.
Portfolio Budget Statement outcome
3.2
The 2017–18 PBS outlines the ACIC's single outcome (Outcome 1) that requires the ACIC 'to make Australia safer through improved national ability to discover, understand and respond to current and emerging crime threats and criminal justice issues, including the ability to connect police and law enforcement to essential criminal intelligence, policing knowledge and information through collaborative national information systems and services'.
3.3
The PBS notes that the ACIC is an agency that:
… works across national boundaries to provide national policing information systems and services and to bring together Commonwealth, state and territory government partners from law enforcement, intelligence, regulatory and other agencies to achieve the ACIC’s outcome of making Australia safer. Every activity the ACIC undertakes has an intersection and linkage with another agency or organisation (including some international partners and private industry).
3.4
The ACIC noted in its 2017–18 Annual Report (annual report) that overall achievement against the outcome 'to make Australia safer' is not always within the agency's direct control or influence. While the agency contributes to this shared outcome, disruption and prevention activities, such as arrests or policy reform, are undertaken by other agencies.
Key performance criteria
3.5
The ACIC performance criteria for 2017–18 are linked to its PBS outcome through the Corporate Plan 2017–18 to 2020–21 and Strategic Plan 2016–21. These two plans categorise the ACIC's approach to its work and related performance criteria into four themes:
3.6
The annual report qualified its performance measurement by highlighting the interrelated nature of its activities. The annual report also noted that while some of the ACIC strategies are short-term and some are long-term, the agency is still required to reflect on its contribution in the reporting period, even though some of its work may not yet have resulted in a tangible response or outcome.
3.7
The report also noted that the intended benefit of the ACIC work is informing other agencies’ responses to crime threats; however, information is often not consistently or routinely recorded or made available by other agencies. Still, the ACIC is able to collect performance data on how joint activities have achieved results against shared outcomes and its contribution to these outcomes. It can also measure the quality, volume of data, and reliability of its information systems.
3.8
Table 3.1 outlines the ACIC performance criteria and overall result for each criterion for the 2016–17 and 2017–18 years.
Table 3.1: ACIC performance criteria results
|
Theme
|
Performance criteria
|
Result
2017–18
|
Result
2016–17
|
|
Discover
|
Discover 1: The picture of crime impacting Australia is improving because the ACIC is discovering crime threats, vulnerabilities, patterns, methods and trends previously unknown.
|
Met
|
Met
|
|
Understand
|
Understand 1: The understanding of the picture of crime impacting Australia is increasingly more comprehensive, integrated and relevant.
|
Met
|
Met
|
|
|
Understand 2: The understanding of the picture of crime impacting Australia is increasingly used to guide strategies and responses to crime.
|
Met
|
Met
|
|
Respond
|
Respond 1 (Prevent): The ACIC better informs and influences the hardening of the environment against crime.
|
Met
|
Met
|
|
|
Respond 2 (Disrupt): The ACIC is conducting investigations and intelligence operations, and disrupting, disabling and dismantling serious and organised crime.
|
Met
|
Met
|
|
|
Respond 3 (Protect): ACIC partners are better informed and enabled to undertake policing and community safeguarding activities through access to national information systems and services.
|
Met
|
Met
|
|
Connect
|
Connect 1: Existing ACIC systems and services are accessible, used and reliable.
|
Partially
met
|
Met
|
|
|
Connect 2: The delivery and implementation of new and enhanced ACIC systems and services satisfies the needs of stakeholders and users.
|
Partially
met
|
Not met
|
|
|
Connect 3: The ACIC is sharing increasing volume, breadth and formats (mediums, platforms) or criminal intelligence and information, police information, and other relevant information.
|
Met
|
|
|
|
Connect 4: The ACIC builds, coordinates and maintains strong and collaborative relationships with domestic and international partners.
|
Partially
met
|
Partially
met
|
Source: ACIC, Annual Report 2017–18, pp. 21–56.
Analysis of performance results
3.9
Wherever possible, the ACIC's performance assessment incorporates a mix of quantitative and qualitative information along with the results of its annual stakeholder survey. The annual report provides an analysis of each performance criterion and a breakdown of stakeholder survey results.
3.10
As demonstrated in Table 3.1, the ACIC met in full 7 of its 10 performance criteria for the 2017–18 year. The annual report noted that the ACIC intends to work on areas for improvement, including stakeholder engagement, project management and delivery, timeliness, and clearly articulating its strategic direction and role.
3.11
The ACIC's general performance for the 2017–18 year received an average overall rating of 6.6 from surveyed stakeholders (from a scale of 1 to 10). This is similar to its 2016–17 result where the ACIC received an average of 6.8 (this represents the agency's benchmark as it was its first year in operation).
3.12
In its overall analysis of its performance, the ACIC noted it had achieved a consistent level of performance in meeting three of its four themes (Discover, Understand, and Respond). The committee has selected a sample of notable results which demonstrate the ACIC's work and effectiveness, as below.
Discover 1
3.13
In 2017–18, the agency's work assisted in uncovering a range of previously unknown threats, including criminal syndicates and identities, links between cybercrime campaigns, threats to Australia's border, and new drug manufacturing techniques. The ACIC noted that new discoveries had continued at similar levels to previous years and covered a broad range of crime themes.
Understand 2
3.14
The ACIC's intelligence guided several strategies and responses to crime in Australia in the reporting period, including:
the establishment of the National Illicit Tobacco Taskforce led by the Australian Border Force;
the Department of Home Affairs' approaches to illegal labour hire syndicates, visa migration fraud, and visa fraud in the education sector; and
14 operations conducted in locations across Australia based on data from the National Wastewater Drug Monitoring Program.
Respond 2
3.15
The ACIC conducted and produced effective investigations, operations and intelligence that assisted to disrupt serious and organised crime. In the period, the ACIC contributed to disruption of 22 criminal entities, which included 19 ‘significant’ and three ‘severe’ disruptions.
Partially met criteria
3.16
The ACIC only partially met three of its four Connect category criteria in the 2017–18 year. The agency determined that it had not fully met these criteria for a number of reasons, outlined below.
Connect 1—accessibility, usability, and reliability of the ACIC's existing systems and services
3.17
The ACIC determined the percentage of time that systems were available and in turn measured these results against previous years and an agreed benchmark. The annual report explained that availability for three of the ACIC's systems that support frontline police and other partners (the National Police Reference System, the National Child Offender System, and the National Police Checking Service) fell below the benchmark required by stakeholders, mostly due to two outages that occurred in April 2018. The ACIC undertook immediate remedial work as well as steps to ensure the issues would not reoccur.
3.18
The ACIC also surveyed stakeholders about their views on its systems and services. Similar to last year's results, only 67 per cent of those surveyed agreed that 'the ACIC's national policing system and intelligence systems were reliable'. However, 91 per cent of stakeholders agreed or strongly agreed that the services and systems were of value or great value to their business area. As a result, the ACIC determined that it had partially met the criterion.
Connect 2—delivery and implementation of new and enhanced ACIC systems and services
3.19
The ACIC delivered and progressed a range of services and systems in the period and it paused planned work on the National Orders Reference System and Australian Cybercrime Online Reporting Network (ACORN) in response to the changing needs of stakeholders.
3.20
The only method used to measure whether systems satisfy the needs of stakeholders is through the ACIC stakeholder survey. The annual report showed that only 61 per cent of stakeholders surveyed agreed that 'new and enhanced ACIC systems are being developed and improved' and only 45 per cent agreed that the 'ACIC's systems meet their organisation's criminal information and intelligence needs'. The results represent a decline from the 2016–17 year (65 per cent and 54 per cent respectively) and were attributed to the decision to close the Biometric Identification Service project (discussed in Chapter 4). As a result, the ACIC determined that it had partially met the criterion.
3.21
With the restructuring of the project management office, the ACIC plans to include post-implementation feedback, analysis, and benefit studies to augment future performance measurement. The ACIC also identified a range of areas for improvement, including that it needs to increase communications with stakeholders and manage engagement at the portfolio level as well as manage projects more effectively. It has established a Technology Collaboration and Coordination Unit to better manage stakeholder relationships and the Enterprise Program Management Office has been working to refine practices. Among other things, the ACIC pledged to deliver smaller work packages more regularly and earlier, hold regular meetings with stakeholders and improve escalation paths. While the ACIC is strengthening its governance of all projects, it noted it may take time to shift stakeholder perceptions.
Connect 4—building, coordinating, and maintaining strong and collaborative relationships with partners
3.22
In assessing its performance of this criterion, the ACIC considered its ongoing relationship building activities specific to projects and task forces, collaboration in developing new services, and stakeholder survey results.
3.23
The ACIC collaborated in a range of joint taskforces, strategies, and projects and expanded its partnerships during 2017–18. Among other things, the agency undertook 25 short-term and five long-term deployments to 15 international locations to assist joint operations, continued involvement with the Five Eyes Law Enforcement Group, and joined the Joint Chiefs of Global Tax Enforcement alliance.
3.24
Although the ACIC collaborated with partners across a range of activities, only 51 per cent of stakeholders surveyed agreed that the ACIC collaborates with their organisation as much as it should. This result is similar to the 2016–17 year (54 per cent). The survey result was assessed by the ACIC as being linked to delivery of its new and enhanced systems and services, and explained it has taken a range of remedial and preventative steps to improve its results as discussed above in Connect 2.
3.25
The ACIC also revised its performance criteria for 2018–19 which is expected to better measure collaboration. The 2018–19 performance criteria now have three Connect criteria rather than four. The ACIC combined Connect 2 (delivery of new systems) and Connect 4 (maintaining collaborative relationships with partners) so that Connect 2 now reads:
Through effective collaboration, enable the delivery and implementation of new and enhanced ACIC systems and services that satisfy the needs of stakeholders and users.
Committee comment
3.26
The committee acknowledges the ACIC's results achieved in 2017–18 and appreciates the difficulty of measuring performance when the results of the ACIC's work are often not directly reported on, made available to the agency by partners, or seen until after the reporting period.
3.27
The committee is pleased that the ACIC has identified areas for improvement including stakeholder engagement, project management and delivery, timeliness, and clearly articulating a strategic direction and role. The committee notes that the agency continues to implement meaningful changes to its processes.
3.28
With regards to the three partially met criteria, the committee is pleased that the ACIC undertook immediate work to rectify systems outages and worked to prevent recurrence of the issues. In relation to Connect 2, the committee appreciates the difficulty in relying solely on the results of the stakeholder survey to measure performance. The committee notes that the agency has undertaken a range of work to improve its processes. The committee looks forward to seeing the results of improved stakeholder and project management. It is also interested in seeing whether incorporation of post-implementation feedback, analysis, and benefit studies augments results.
3.29
The committee acknowledges the considerable collaborative work the ACIC has undertaken throughout the 2017–18 year. It notes that the Connect 4 result is linked to partly met Connect 2 criterion and that this is the second consecutive reporting period that the agency has not fully met these two criteria. Stakeholder survey results of 54 and 51 per cent for the last two reporting periods are disappointing. The committee anticipates that the ACIC's results will improve after it has had time to demonstrate changes to stakeholder and project management.
3.30
The committee notes that the ACIC has combined its Connect 2 and Connect 4 criteria for the 2018–19 year. While the committee did not consider this change in its examination of the 2017–18 annual report, it may consider the merits of this change in its examination of the 2018–19 annual report.