Chapter 6


Chapter 6

Case Studies

Introduction

6.1        This chapter addresses the circumstances of four quite different case studies to provide a practical insight into the nature of the issues and challenges posed by the digital transformation of government services and administration.

6.2        Each of the case studies is intended to showcase issues arising with different aspects of digital transformation. The case studies are:

  • The Australian Taxation Office unplanned systems outages of the Storage Area Network in 2016 and 2017. This case study explores questions of risk sharing in contractual arrangements for business as usual ICT infrastructure.
  • The Department of Human Services' replacement of the child support payments computer system, Cuba. This case study explores the challenges of replacing legacy infrastructure.
  • The controversy caused by the automated Online Compliance Intervention letters issued to welfare recipients also known as "robo-debt". This case study explores problems arising from failures at a policy design and selection stage.
  • The Welfare Payments Infrastructure Transformation (WPIT) project to replace Centrelink's aging ISIS computer operating system for welfare payments. This case study explores new, more sophisticated contractual arrangements for sharing risk with commercial partners and delivering value for taxpayers.

6.3        The committee has expressed views about the circumstances of each of the case studies below. The committee's conclusions about broader systemic implications of the case studies, however, are included in the committee’s views in chapter 1.

Australian Taxation Office—Unplanned Systems Outages[1]

6.4        In December 2016, and throughout 2017, the Australian Taxation Office (ATO) experienced a series of 'unplanned systems outages' due to hardware failure of its Storage Area Network (SAN). These outages were reported as having a significant effect on the ability of the public and tax professionals to engage with the ATO. A summary of the outages can be found below in Table 4.1:

Table 4.1: A summary of the ATO's unplanned outages during 2016–2017.

Start date of the system outage

Duration of the outage

Services affected

Cause of the outage and delay to restore services

12 Dec 2016

10 days

All ATO systems
  •                            Storage area network hardware failure.
  •                            Inadequate monitoring.
  •                            Recovery tools stored on failed storage area network.

2 Feb 2017

5 days

All ATO systems, website running intermittently
  •                            Incorrect storage hardware installation.

22 Jun 2017

3 hours

All ATO systems
  •                            Hardware failure on a server, leading to Active Directory domain controller failure.

5 Jul 2017

5 hours

All ATO systems
  •                            Applications running incorrectly.

25 Sep 2017

6 hours

All ATO systems, website running intermittently
  •                            Applications running incorrectly.

Background

6.5        In 2010, the ATO had signed a five-year contract with Hewlett Packard Enterprises (HPE) (now rebranded DXC) to provide centralised computing services, with a commencement date of July 2013.[2] In July 2017, the ATO provided a brief description of the ICT services that were contracted:

In December 2010 we signed a five‑year contract with HPE for Centralised Computing (CC) services. After a stabilisation period to ensure the proper transition from an earlier arrangement, the five‑year contract term commenced in July 2013. The scope of the CC services offered to the ATO includes our large processing systems (systems of record), systems of client engagement (portals), data warehouse and internet gateway services.

At the beginning of 2015, a sourcing, design and implementation process commenced in relation to the ATO's storage area network (SAN) solution. HPE recommended the installation of a state‑of‑the‑art HPE 3PAR SAN to replace the existing EMC Corporation SAN. This was on the basis that the 3PAR solution created a more flexible storage environment that would better optimise costs was supported by HPE operating procedures and technical expertise.

This was agreed to by us, and the installation of the new 3PAR SAN was completed in November 2015.

We engaged HPE to provide turn‑key IT solutions, whereby HPE designs, owns and operates computing infrastructure and provides services to the required ATO standard. Under this turn‑key operation, ATO IT staff have no direct access to the SAN technology operated by HPE. Instead, we rely upon HPE to provide a full service. To enhance and coordinate the work of our IT contractors, the ATO also contracted with Leidos Holdings, Inc. (Leidos) as service integrator. Leidos operates a virtual dashboard over myriad ATO IT systems, and provides a problem management process should issues arise with parts of our IT infrastructure.[3]

6.6        Analysis of the centralised computing network data for the six months preceding the initial incident in December 2016 indicated a number of potential issues. From May 2016, at least 77 events relating to infrastructure components that were later found to have failed in the December 2016 incident were logged in the Leidos' incident reporting mechanism. In addition, at least 159 alerts were recorded in monitoring and management logs.[4]

6.7        Many of the system outages arose primarily because of failures in the centralised computing services provided by HPE/DXC. The design and build of the centralised computing network emphasised performance over other critical factors such as system stability, resilience and cost, with the result that there was insufficient resilience. Furthermore, the recovery process was extended as the tools required to restore ATO services were stored in the failed SAN, and were therefore inaccessible because they were dependent upon access to the failed SAN.[5]

6.8        Another significant factor was the failure in communication between the ICT provider (HPE/DXC and Leidos) and the vendor (ATO). It is not clear from the ATO report whether responsibility to inform the ATO of the infrastructure issues rested with HPE or Leidos. It is clear however that the ATO was not made fully aware of the significance of the continuing trend of alerts, nor the broader systems impacts that would result from the failure of the system.

6.9        The Australian National Audit Office review of the ATO unplanned outages found that the ATO's response to the unscheduled outages found inadequacies in business continuity management planning relating to critical infrastructure.[6] An independent review by PwC has found that the ATO had failed to plan for an incident of the nature and scale of that which the ATO experienced. Moreover, a level of risk continued to exist due to the absence of definitive evidence on the conditions that led to the technical failures in the first place. Investigations of the root causes of the technical failures are still yet to be completed by the service provider.[7]

DTA's response

6.10      Dr Lesley Seebeck, Chief Investment and Advisory Officer, Digital Investment Management Office, Digital Transformation Agency (DTA), advised the committee that the DTA had no involvement in the ATO outages of December 2016 and February 2017, on the basis that the DTA does not have oversight of operational processes; and that the issue was an agency responsibility.[8] Mr Ramez Katf, Second Commissioner and Chief Information Officer, Enterprise Solutions and Technology, ATO, advised the committee that he had briefed Mr Gavin Slater, Chief Executive Officer of the DTA after Mr Slater joined the DTA in June 2017, after the two outages had occurred.[9]

The outage

6.11      Mr Katf confirmed that the SAN outage was caused by a fault in the fibre optic cabling, said to be an 'unprecedented event'. He advised that the faulty hardware was less than one year old. Mr Katf told the committee that it was highly reputable technology provided by a highly reputable provider with whom the ATO has a long standing relationship.[10]

6.12      Mr Katf confirmed that the system lacked resilience, and agreed with the proposition that the outage implied a single point of failure. Mr Katf stated that in developing the infrastructure the ATO believed resilience capability had been built into the technology. He advised that the construct taken in relation to the piece of hardware that failed due to multiple failures within the system, which impacted on all the ATO systems, noting that the 'resilience factor was, in fact, built in, but failed'.[11]

6.13      Mr Katf stated that he had not yet received a definitive answer as to the cause of the failure.[12]

Overview of the ICT purchasing decision

6.14      Mr John Dardo, Chief Digital Officer and Deputy Commissioner, Digital Delivery, Enterprise Solutions and Technology, advised that the initial procurement decision was to purchase ICT equipment, services or capability without having to design from the ground or build the system themselves:

We didn't want to build the telephony infrastructure, the network or the data centre. We didn't know how to build a data centre. It was about using industry expertise with leadership from within the office.[13]

6.15      The ATO advised that decisions concerning the sourcing of the ICT infrastructure were managed internally. Strategic advice was obtained from external experts, which were factored into the approach to market. All key decisions including the procurement approach, evaluation outcomes and contract execution were the responsibility of the ATO.[14]

6.16      The ATO further advised that the ICT Sourcing Program was itself managed primarily by ATO employees, supported by external engagements for specialist advisor expertise. The objective being to maximise the use of appropriately skilled internal resources, supplemented by external resources where necessary. The ATO does not retain the specialist adviser skill set as an enduring capability due to the potentially long periods between utilising these skills, whereas ongoing ATO employees are utilised in the governance and management areas.[15]

6.17      Mr Katf advised that for the architecture components the ATO service providers would undertake the primary design capability and the ATO would provide assurance, review, design and sign-off of those items.[16]  Mr Katf further advised that the ATO has approximately 200 IT staff whose job is to make sure the ATO has an integrated design that goes from the end-user right down to the technology componentry. [17]

6.18      Mr Dardo said that the broad suite of performance evidence in the ATO suggested that the ATO got the balance right between internal and external expertise in its approach to the ICT sourcing task.[18] Mr Katf observed that it was not unusual to engage external advises to provide technical capability, but agreed that the danger is in the balance between ATO staff and external consultants:

I am very confident at the moment. We tend to bring in advisers—but, as you said, on short, sharp capabilities—to provide us with advice, but we supplement that with our own people to traverse the questions and to make sure that we maintain that capability in-house.[19] 

6.19      In response to a recommendation of the ATO system report to enhance the ATO's IT capability pertaining to infrastructure design and implementation planning, Mr Katf advised that a new team has been established to strengthen the infrastructure and design capability, and actions have already been taken to improve the ATO's governance and design capabilities. A skills gap analysis has been undertaken and identified further recruitment requirements. To augment the existing capability, the ATO has engaged an external consultancy for the period from April to June 2018, four external contractors for the period from February to June 2018, and eight APS staff for the period from February to August 2008.[20]

Contracting arrangements

6.20      Mr Katf confirmed that the original contracts for the infrastructure were entered into in 2009 and 2010, and that the contracts related to three infrastructure service providers:

  1. DXC [formerly HPE] for the centralised computing covering the mainframe, midrange computers and data centres;
  2. Leidos to manage the end-user PC devices, and service management capability, and in that sense in essence the integrator across the technologies; and
  3. Optus to provide the telephony.[21]

6.21      Mr Katf advised the 2009 DXC contract was amended in 2015 to provide revised data storage arrangements.[22] All providers' contracts are held directly with the ATO. Leidos provided a broader capability that allows Leidos to assist the ATO in managing across all vendors, and the ATO'S own internal capability.[23] Mr Katf confirmed that the current contracts are due for renegotiation in 2018.[24]

6.22      Mr Katf advised there was a separate internal group responsible for contract performance to manage the responsibilities and accountabilities of the ATO's vendors. He advised:

We are, in essence, the systems integrator, because we have the responsibility because we also own the business applications that sit on top of the [infrastructure]. This was a conscious decision, I think, that we are architected our own solution.[25]

6.23      Mr Katf advised that the ATO were the managers of service level agreements which allowed for penalties if key performance measures were not met. Each month the ATO measures the performance of the service providers against specified performance measures. The contract with each service provider included the thresholds for both the expected and the minimum performance level. Where the service provider fails to achieve the minimum performance level, the ATO at its absolute discretion can apply service credits (known under the contract as Performance at Risk Amounts). The service credits are a defined percentage of the monthly invoice that have been agreed as having regard solely to the reduction in value to the ATO of the services which have not been performed.

6.24      Where the service provider does not achieve the minimum performance level, and the service credits have been allocated by the ATO, these are accrued until the end of the contract year where, at the ATO’s discretion, these amounts can be 'earned back' by the service provider.[26]

6.25      Mr Katf advised that penalties were imposed on DXC arising from the outages, but based on both internal and external legal advice no penalties were imposed on Leidos as the assessment was that Leidos did not breach any of its obligations, but it was very clear that DXC had not met the service levels. The settlement with DXC resulted in the ATO recouping its key costs, as well as providing the ATO with higher grade IT equipment relating to data storage.[27] 

6.26      The ATO advised that it chose to not apply the standard contract performance framework to the contract that led to the outages of December 2016 and February 2017 on the basis that the contract performance framework was not intended to deal with an outage of the magnitude of the SAN outages. The contract contained other commercial remedies that are intended to deal with situations such as these, which were utilised.[28]

6.27      The ANAO noted that the ATO 'does not have measures at the corporate or strategic level to enable a confident assessment of whether risks exceed tolerances' and found that there was significant variation in the extent to which tolerances were specifically contracted for.[29] The ANAO concluded that this lack of clarity around contract risk management could impact future procurement activities unless changes were made:

The ATO's ICT infrastructure continues to be modified in response to demands for online services, and the availability of new technologies to support digital platforms and address risks and issues with legacy ICT systems. Use of new technologies is resulting in the ATO entering into different types of contracts with service providers. In 2018, the three bundles of major ICT contracts will be due for renewal. The combination of these events provides the ATO with an opportunity to reassess its ICT service measurement approach, and where possible implement common approaches, at least in terms of reflecting tolerances that align with the ICT outage service standards that the ATO has committed to develop.74 Such an approach would support the ATO in its efforts to use digital technology and online services effectively and efficiently in the administration of the taxation and superannuation systems

Two particular matters to consider are:

  • revise the service measurements applying to the Amazon Web Services cloud service contract that does not include service level provisions. This contract exposes the ATO to contractual and operational risks in the absence of measurable service levels.
  • to the extent possible, align service measurements arrangements for services sourced through ATO procurement processes, and those obtained through whole-of-government and shared ICT procurement options.[30]

6.28      The ANAO recommended that the ATO enhance its capability in relation to resilience and risk.

Importantly, at an entity-level, greater definition is required as to how the ATO engages with key vendors, supported by greater analysis and monitoring of arrangements, including periodic reporting to the ATO Executive. In this way, the ATO will better define and achieve strategic value from vendors, with better visibility and control of the breadth of, and reliance upon, vendor arrangements.[31]

6.29      The ATO accepted the recommendations in this report noting that improvements were already underway.[32]

Increase in contracting activity in 2009

6.30      The Austender website showed that in 2009, there was a significant increase in tenders for ICT services.[33] Many of these contracts appear to be for short term contracts (that is, less than 12 months) for roles relating to solution design, systems architecture, SAP and mainframe specialists. It is not clear whether some of these roles related to the management of the increasing number of contracts. Furthermore, it is not clear how the corporate knowledge of these contractors was retained by the ATO when contractors left the organisation.

Committee view

6.31      In many ways, the ATO represents one of the more advanced examples of digital transformation. Millions of Australians submit their tax returns online. Businesses, including tax agents, access ATO data everyday through digital portals. 

6.32      The corollary of the penetration of digital into the work of the ATO, however, is that service delivery was increasingly dependent on its ICT infrastructure and contractors. A failure in either would mean that the ATO would be incapable of delivering the standard of service that it had promised, and that its end users reasonably expected. Ultimately, this is exactly what occurred. The committee considers the sheer volume of outages suffered by the ATO was largely unprecedented and entirely unacceptable. It was seriously disruptive to the general public.

6.33      It is the committee's view that the importance of ICT and digital services to the ATO’s business model was not reflected in the types of contracts it had with its ICT service providers.

6.34      This may partly be a legacy issue—contracts were entered into some years ago, and the volume and importance of digital services may have increased since then. The committee considers, however, that this is now a foreseeable risk that agencies and departments should budget for when making procurement decisions that are intended to last a number of years into the future.

6.35      The committee considers, however, that part of the issue with the ATO's contracting was a lack of awareness of precisely what level of service was being contracted for, and the agency’s consequent degree of exposure.

6.36      The committee notes the ANAO's conclusion that the ATO 'does not have measures at the corporate or strategic level to enable a confident assessment of whether risks exceed tolerances'.[34]

6.37      This is no longer a viable approach. The core business of much of the ATO is not delivery of tax services, but digital delivery of tax services. This is not just a question of delivery mechanism, but is built into the nature of the service that users have come to expect, that is, 24 hour access on demand.

6.38      The committee recognises that different standards of service are available at different price points. Ultimately, it is a procurement decision for each department to make about what standard of service they require. The committee considers, however, that this decision should be made with a full and complete understanding of the department’s risks and needs.

6.39      This requires a more digitally educated workforce that currently exists.

Child support system replacement project

Background

6.40      The Australian Government child support IT system is known as Cuba. Cuba processes payments of '$3.5 billion from separated parents to financially support the welfare of over 1.2 million children'.[35]

6.41      In the 2013–14 Budget, the Australian Government introduced a budget measure to replace DHS's Cuba. Funding for the project would be drawn from existing departmental resourcing.[36] The replacement of Cuba was considered to be necessary because the existing system was 'getting close to the end of its useful life' and the new system would provide 'better support for staff and separated families'. The budget measure indicated that the new system would be introduced by December 2015.[37]

6.42      An expression of interest process to replace Cuba was advertised in July 2013 on Austender.[38] From the outset, industry observers were sceptical of the tender's design:

Considering the work entailed, the projected timeframe for the project appears quite tight. DHS has entered an expressions of interest phase for the development work and plans to issue a formal request for tender document for the project in September or October this year, after short-listing a number of companies for the initiative.

It then expects to deploy the replacement project by mid-2016, giving it likely only about two and a half years to do so, while transitioning all existing customer data onto the new platform by the end of that year. It then plans to implement additional enhancements to the new system by the end of 2018, with a view to more fully supporting current government legislation and policy.[39]

6.43      Furthermore:

Then, too, there are already indications that the CSS overhaul is going to have problems. Keen observers will have noted the following paragraph in its EOI document last week:

"Development of the child support system will require close liaison with SAP to redevelop base modules and avoid bespoke modification to aid efficient development of the new system and ensure upgrade pathways for SAP solutions are not compromised."

Is DHS really saying that it wants its project partner to work with SAP on re-developing SAP modules to fit its own needs, and then re-integrate those modules into the mainline SAP codebase, so that future upgrades aren’t a problem? Some would call that a rather 'ambitious' approach. Convincing a mega-vendor to do anything like this is always going to be a headache. And especially for a project as small (on global terms, $100 million is nothing) as the CSS overhaul.[40]

6.44      By February 2014, the tender process had concluded and the tender was awarded to Accenture. The then Minister for Human Services, Senator the Hon Marise Payne suggested that government's expectations were that in-house capability would be delivered as part of the tender:

Accenture and SAP will assist in building the replacement system while ensuring that the department is left with a skilled, in-house workforce able to maintain the system into the future, a reduced cost to the taxpayer.[41]

6.45      The replacement system known as PLUTO was supposed to be finalised by mid-2016. This deadline was not met. By the start of 2017 there were media reports of significant difficulties.

DHS claimed in 2013 that it could have a replacement up and running by December 2015 but more than 12 months after the mooted launch date for the new system, Child Support workers are still using CUBA and are still in the dark on the fate of the replacement project.

In mid-2016 agency staff were promised a stop-gap solution, the continued use of CUBA, but with modern, bolted-on front screens using technology supplied by German tech giant SAP and acting as a "wrapper" for the older CUBA technology.

But Fairfax understands that nothing has come of the promised SAP screens with insiders reporting that a "deathly silence" has fallen over the entire child support system replacement (CSSR) project.[42]

6.46      DHS finally launched the new system in July 2017 with a number of problems becoming immediately apparent:

Child Support Agency staff report the new system, called Pluto, is slower and clumsier than the obsolete technology it was supposed to replace.

Both frontline public servants and the main workplace union has told Fairfax Media that large numbers of CSA public servants being ordered to drop everything to help with "emergency escalations" as the agency's phone lines are swamped with irate clients.

...Now that Pluto is finally operational, one frustrated user told Fairfax the process was a "shambles"

"It's no easier and considerably slower than Cuba while we're left making excuses to customers to calm them down," the insider said.

"We only have about half the information we used to, so it's very difficult telling customers where their payments are, when they can expect to receive payment."[43]

6.47      These issues were initially denied by the DHS, however, it was reported that soon after the initial denials, DHS finally acknowledged that PLUTO was not working as it should:

On June 19, Kate Hay, the general manager of the agency's call centres, wrote to her staff thanking them for their "understanding and commitment" during the roll-out of the new system.

"The executive and I acknowledge that this extremely busy period has meant that some of you feel that you are not on top of your work as you would like to be, please be assured that we realise that you are all doing your best to manage this and seeking assistance to manage some aspects which may not be within your direct control," Ms Hay wrote.

"We also note that there have been more escalations required recently to meet our demand which, can seem disruptive to your day."[44]

6.48      At the committee's Sydney hearing, Mr Osmond Chiu, Research and Policy Officer at the CPSU informed the committee of his members' experience with the new system:

There have been a range of issues with that, which occurred last year. It's a new system that was introduced, and many people who rely on child support payments couldn't actually use the system. This is a new system, and they had to drag child support staff from their usual work to answer the phones because the system didn't work.[45]

6.49      At the recent Senate Community Affairs Legislation Committee's Additional Estimates, DHS staff told the committee that staff have to use both the old system (Cuba) and the new system concurrently. As Ms Maree Bridger, General Manager, Child Support and Redress Division, DHS explained:

[Staff] need to use both systems, but, because everything that Cuba does is not replicated in Pluto, at times they will need to enter some things into Pluto and at other times they will need to enter them into Cuba, and there are some teams that do complex functions that are solely undertaken in Cuba.[46]

6.50      Mr Charles McHardie, Acting Chief Information Officer at DHS, explained how the systems currently interact with each other:

Pluto is the front end. It's collecting the data. It pushes it into Cuba. It does the calculations and remains the system of record to provide that payment assurance.[47]

6.51      Similarly to the Welfare Payments Infrastructure Transformation Program (WPIT), Mr McHardie pointed out that the replacement of Cuba would be the last step in the replacement program.[48] The WPIT case study is discussed later in the chapter.

DTA response

6.52      Dr Seebeck of the DTA advised the committee that the Cuba replacement project had been captured in the DTA's digital investment review data collection, but the project is not within the DTA's 'engage category' because the project was nearing its end following re-scoping. Furthermore, the DTA had confidence in the department that action was being taken. She observed that the Cuba project faced the types of problems that were not uncommon when dealing with legacy systems. She advised that the project commenced before the mandated requirement that all projects follow the DTA's Digital Service Standard.[49]

DHS response

6.53      Ms Maree Bridger, General Manager, Child Support and Redress, DHS advised that an independent review in 2009 determined that Cuba was nearing the end of its life. The 2013–2014 budget allocated $102.3 million to replace the legacy system (Cuba) which did not automate all child support processes; with some processes being completed manually.[50] The allocated budget was spent over the years 2013–2016[51] with the $102.3 million being funded internally by the department.[52] The government's plan was that once Cuba was replaced, Cuba would then be enhanced to accommodate those transactions that were currently being done manually. As at March 2018, not all existing legacy processes have been transferred to the new system.[53] 

6.54      Ms Bridger advised that the allocated $102.3 million had been spent on progressing the Cuba project to its present state of development.[54] Ms Renee Leon, PSM, Secretary, DHS, advised the committee that the expenditure has delivered benefits. DHS told the committee that it has made considerable progress with the child support system redesign. However, Ms Leon noted that the Cuba replacement program was more complex than had been anticipated, requiring more work from the department. Moreover, the department has obtained new technologies which are now being utilising to build a better system.[55]

6.55      Mr McHardie explained that Deloitte has been engaged to assist DHS with an evaluation of the rollout of Pluto. Deloitte will assist in assessing the approach DHS has taken to date and with the prioritisation of the next steps. Mr McHardie indicated that the Deloitte report will assist the Department to clarify the project.[56]

6.56      On the engagement of Deloitte to clarify the strategy that would lead to the implementation of Cuba and Pluto, Ms Leon said:

At every Estimates the plan will become clearer...I don't think we said it was an endless project. We said we're in the process of scoping now. We don't at this point have a timeframe for it, but we don't intend to run it an unbounded fashion.[57]

6.57      Cuba comprises a 'front-end' function, a generic term which refers to the staff interface with a computer screen to process information, and a 'back-end' function, which is the calculation engine that assists with, in Cuba's case, assessing the quantum of child support payments, and the storage of data in a legacy database.[58] In July 2017, Pluto replaced the front-end function of Cuba, but Cuba continued to undertake the back-end functions.[59] Mr McHardie advised that it still remains necessary to replace the Cuba back-end legacy system.[60]

6.58      Mr McHardie confirmed that in 2016, a decision was taken to move from an out-sourced delivery methodology to an in-house delivery approach led by the Chief Information Officer Group. The decision to vary the approach originally agreed upon was to enable the department to leverage off work being done for the Welfare Payment Infrastructure Transformation (WPIT) program in relation to the Centrelink processes. In anticipation of the WPIT program, the department started to move the front-end staff-facing processing screens online and mobile component that the public view into a new pattern using SAP technology that was being built in-house. In April 2016, that aspect of WPIT was leveraged in order to build Pluto. [61]

6.59      Mr McHardie advised that the work undertaken by contractor Icentia between 2014 and 2016 had not been abandoned. Incentia had undertaken an examination of how Cuba was constructed—the functional building blocks and all the functional requirements in Cuba that would need to be replaced. This process addressed how SAP, the customer relationship management system could be utilised to replace Cuba in its entirety Pluto is built of a SAP platform.[62]

6.60      Mr McHardie acknowledged that the contractors undertaking the initial examination of Cuba found it to be more technologically complex than they had envisaged at the outset, however, a lot of the discovery phase work allowed the department to leverage as the project went forward.[63]

6.61      Mr McHardie advised that the selection of SAP was a technical decision. He stated that the department is using the SAP platform for all of the staff-facing capability for the Centrelink Master Program, Mr McHardie advised that the decision to continue with the SAP platform was based on work undertaken in the early stages of the WPIT Program. Mr McHardie noted that DHS had invested heavily in its workforce in SAP technology and has built up a very large skill set with SAP. He also noted that DHS has been using the SAP business suite across many of its ICT builds, both for the department and on behalf of other departments and agencies.[64] 

6.62      Mr McHardie further advised that project methodology had evolved in recent times, adopting a service delivery design process which involves including both members of the public and the department's processing officers so that for large systems DHS can build capability in multi-disciplinary teams and iterate the process starting with a prototype. He advised that Pluto was now being built by multi-disciplinary teams on the service delivery model.[65] Ms Bridger confirmed that the service delivery methodology was not widely used at the time the Cuba project started.[66]

6.63      Mr McHardie further explained that DHS is focussing on change management. It has introduced an new general manager of change management to address change at the staff and enterprise level, including training and communication issues, and a new Chief Citizen Experience Officer to look after change from the public perspective. He considered DHS had made significant improvements in the way it is organising itself to delivery large scale ICT projects, such as WPIT.[67]  

Committee view

6.64      The replacement of aging infrastructure is not a particularly transformative project—it is part of the usual and necessary business required to enable the agency to continue to undertake its work. It is also the type of project which has been, and will need to be, replicated across departments and agencies over the coming years.

6.65      The replacement of Cuba is far from an exemplar. It is a project that was supposed to be completed by late-2015, then mid-2016, was supposedly completed in mid-2017, ran into immediate problems, and then was suspended in mid-2018.

6.66      The committee considers that the consequences of this failure are serious.

6.67      The end users of technology are not always the public—they are quite often departmental employees. That does not make user-centredness and ease of use any less important. The failed replacement of Cuba demonstrates why.

6.68      The committee heard evidence that the partial roll out of Pluto meant that staff were required to use both the old and new systems. These systems were not interoperable. Some functionality remained on the old system whilst some resided on the new. Staff were required to manually transfer data between them. Training on the new system was complex and took extended periods of time.

6.69      The impacts of this on staff are regrettable. ICT changes should not make work needlessly more difficult. However, the impacts on public servants also flow through to the service that is able to be delivered to the public. The committee heard evidence that staff were pulled from answering hotlines in order to receive training. The need to use two systems substantially increased the amount of time it took to undertake any task. In the absence of further resourcing, a backlog built up.

6.70      This was all happening within the Child Support Agency—a government function designed to help often vulnerable people and their children, at times of conflict, need, and distress.

6.71      A substantial sum of taxpayer funds have been sunk into a project that has been suspended indefinitely.

6.72      The committee appreciates that there are unexpected difficulties that arise with replacing aging ICT infrastructure such as Cuba. However, departments and agencies undertaking projects like this should expect that unexpected difficulties will arise.

6.73      The scoping of the project was criticised by industry observers as being unduly optimistic when it was first released. The committee has no doubt that that this is true.

6.74      There are incentives inadvertently built into both the department’s internal budgetary processes and the tender processes to present a project as being cheaper and easier to complete than it may actually be. The replacement of Cuba demonstrates that succumbing to these incentives has real consequences.

Online Compliance Intervention (Robodebt)

Background

6.75      The Online Compliance Intervention (OCI) process applied an automated system to a data-matching process comparing income data held by the Australian Taxation Office with the income payments data of Centrelink in an effort to identify discrepancies as the basis to recover overpayment of welfare support payments from Centrelink and former Centrelink recipients.[68] Data-matching between Centrelink and the ATO has been undertaken for approximately 20 years. Prior to the introduction of OCI, the process of checking the ATO lump sum income records against [Centrelink's] fortnightly income records', identifying where someone has been overpaid was undertaken by departmental personnel.[69]

6.76      OCI represented part of a 2016 Coalition election commitment to improve the sustainability of the welfare system by reducing overpayment of income support payments.[70]

6.77      From November 2016, under the OCI, 'where there was a discrepancy between the income declared to the ATO and Centrelink's records, a letter was automatically generated that asked recipients to use an online portal to update their details', in effect, outsourcing the department's role to the 'individual income payment support recipients'.[71]

6.78      In 2016–17, the Government forecast that the program would deliver $3.7 billion worth of savings. However, for the period July–December 2016, the 'department had sought repayment of $300 million worth of purported debts and actually recovered $24 million'.[72]

6.79      In its submission to the committee, the CPSU summarised its view of the OCI, stating:

There are three fundamental failures built in to the OCI. Firstly, the human oversight involved in assessing discrepancies and raising debts has been limited. A second and related flaw is that the administrative cost of managing overpayments has been transferred from the Department to ordinary Australians, with the Department no longer taking responsibility for contacting employers to investigate discrepancies before debts are raised. The business process has been designed to minimise cost to the government by reducing the usual manual oversight requirements and removing employer verification of PAYG anomalies prior to customer contact commencing. The business process design has all but ensured high rates of error in the calculation of debt. Staff have been directed not to fix errors they could clearly identify. Instead they have been instructed to refer customers to online self-service portals in an attempt to transfer the administrative burden of debt recovery onto the customer.

Thirdly, the onus of proof has in effect been reversed, with customers now obliged to investigate alleged discrepancies and provide evidence that an overpayment doesn’t exist – rather than the burden being on government to show that it does.[73]

6.80      In June 2017, the Senate Community Affairs References Committee reported on its inquiry into the OCI and found a number of serious shortcomings with OCI, including:

  • a fundamental lack of procedural fairness at every stage of the OCI program;
  • the lack of procedural fairness disempowered people in dealing with the OCI debt;
  • the department had a fundamental conflict of interest – the harder it was for people to navigate this system and prove their correct income data, the more money the department recouped; and
  • the department did not apply 'best practice'.[74]

6.81      Concurrent to the Community Affairs inquiry, the Commonwealth Ombudsman conducted an own-motion investigation which reported in April 2017.[75] The Government accepted all eight of the Ombudsman's recommendations.[76] A critical recommendation of the Ombudsman was that future letters to income recipients should 'expressly inform individuals [that] if they do not clarify income data, ATO annualised data will be averaged and this may result in a debt'.[77]

The development of OCI

6.82      Mr McNamara, Acting Deputy Secretary, Integrity and Information, DHS, advised the committee that a number of budget measures underlie the OCI data matching process. The first measure in the 2015–2016 budget was the 'Strengthening the Integrity of Welfare Payments', which led to an increase in compliance reviews, initially for the 2015–2016 financial year.[78] The government's decision was that the OCI reviews would be undertaken online in parallel with a manual process.[79] Accordingly, compliance reviews were undertaken manually by DHS staff without reference to an online system; concurrently during the same year the online system was developed. In July 2016, the online system was trialled with a 1 000 person pilot, before being more fully rolled out in August–September 2016.[80]

6.83      In evidence to the committee, Mr McHardie, Acting Chief Information Officer, DHS, advised that the system was built in 2015–2016 as an in-house project.[81] Ms Liz Bundy, Acting General Manager, Integrity Modernisation, DHS, said DHS used a waterfall methodology involving joint design sessions with ICT staff working through the requirements for the project, and then building to those requirements.[82] The analysis and design of the OCI was completed in October 2015 for a start date of July 2016. This occurred before the introduction of the DTA's Digital Service Standards.[83]

6.84      Dr Seebeck advised that the DTA was not involved in the development of the OCI project.[84] Mr Peter Alexander, Chief Digital Officer, Digital Division, DTA advised that in January 2017 the DHS sought DTA advice on how to improve the screens, and received expert advice on user experience and interaction design. Since then the DTA has provided intermittent advice to DHS on items of technology or user research.[85]

The ATO role in data matching

6.85      Mr McNamara further advised that DHS has always coordinated a data matching process with the ATO. The data matching has shown that DHS's capacity to action the anomalies disclosed by the data matching was limited; only a certain number of reviews relative to the significant number of discrepancies between the DHS and ATO datasets. DHS had established a backlog of discrepancies that had not been actioned. The OCI compliance measures were intended to address the backlog of discrepancies.[86]

6.86      The ATO was not involved in the process because data matching with the ATO has been a longstanding process; as such, the OCI project did not change DHS's interaction with the ATO.[87] The existing ISIS[88] operating system applied back-end established rules to data match ATO and DHS records to identify a pool of anomalous records; a further selection process was taken to identify from the pool those people in all probability had been overpaid; the process was not to chase small anomalies.

The OCI letters

6.87      Letters were then sent to those identified seeking an explanation of the anomaly. Where previously recipients of letters would have been required to contact a compliance officer to resolve the anomaly, the OCI letter directed the recipient to an online site to update their financial data. Having entered their data online, ISIS would then undertake back-end [income/debt] calculations. [89]

6.88      Mr McNamara advised that the OCI measure allowed more letters to be sent to potential debtors than was previously the case.[90] Mr McNamara said the online compliance process was predominantly for those with relatively simple arrangements; the OCI letters would not be sent to those with reasonably complex financial circumstances. Also, as an example, Mr McNamara advised that if a large amount of income was entered in one month compared to average weekly earnings, the OCI process would direct the person to telephone the DHS directly rather than using the online system.[91]

DHS' assessment of OCI

6.89      Mr McNamara stated that he was satisfied that OCI met its project intention in achieving considerable savings. He confirmed that the effects of OCI on the community were included in the metrics for the project, observing that it has always been the case that people have been required to advise Centrelink of any change in their circumstances. He said that the fact that the data matching process identified some people who had failed to update DHS of their circumstances did not absolve DHS from the need to assess those welfare recipients.[92] Mr McNamara observed that people react differently to the compliance review process:

...One of the things we've found in the compliance review space, I think it's fair to say, is that people who really don't think they've done the wrong thing and who have genuinely tried to comply with the system do find the idea of talking about their previous history quite confronting. It doesn't really make any difference—the nature of our system. They see it as an integrity issue, and I can accept that. I think that's quite appropriate. If you've been giving us information quite often and you've been doing it quite diligently, for someone to turn up and say, 'I want to essentially audit what you've told me,' can be quite confronting.

In this particular case, there's quite a number of examples we have within the system where, for instance, people quite diligently told us their net pay, but they told us they'd put it in gross pay on our online app. They're diligently telling us the wrong information. Therefore, five years later, when we come along because we now have the capacity to look at the data match and say, 'Hang on—you haven't quite told us the right information', people quite rightly become upset. The difficulty for us is they were overpaid at the time, and some people find this quite confronting as a process. As we've rolled that out, we've understood that nature—that some people are in that category, where other people are in the category of: 'I just needed the money at the time. I've told you the wrong thing. How do I pay it back?' My compliance officers deal with a spectrum of people at the moment, and the online system has to deal with that spectrum.[93]

6.90      In response to the observation that the OCI was only focussed on budget savings—that user's experience was not the primary  purpose of the project—Mr McNamara stated that the issue for DHS compliance has always been the integrity of the welfare system. He said to date DHS had saved $900 million through income matching, and have recovered nearly $270 million. He confirmed that the measures are expected to save $3.7 billion over the period of the project in 2021, and that he was confident DHS will make those types of savings.[94]

Systems design issues

6.91      Ms Liz Bundy, Acting General Manager, Integrity Modernisation, DHS, advised that staff became involved with the rollout of the OCI from May 2015 for a commencement date of 1 July 2015, with approximately 200 staff across five compliance sites being involved. Ms Bundy told the committee that a team of compliance staff located in Brisbane worked with the ICT staff to design the system requirements.[95] Mr McHardie stated that DHS had 136 ICT staff involved in some capacity during the building of the system in the 2015–2016 fiscal year. Mr McNamara contended that operational staff were part of the design process.[96]

6.92      Mr McNamara said that DHS subjected OCI to significant user testing prior to the system being released, not only from an IT perspective, but also from the user's perspective. However, Mr McNamara did agree that some aspects of the communication process could have been clearer on the initial rollout, noting that key changes have since been made to the covering letter and the online system to improve clarity.[97] Mr McNamara advised that in January and February 2017, a number of enhancements and changes were made to the compliance letters to clarify the letters and simplify the language of the letters, on what is now called 'employment income confirmation'.[98] Mr McHardie advised that DHS had involved the DTA in building the employment income confirmation system.[99]  

6.93      Mr McNamara also conceded that a decision was made to not include the OCI dedicated helpline telephone number in the covering letter to welfare recipients. He further conceded that it would have been more useful for people to have had been made aware of the helpline number in the covering letter.[100] Mr Alexander observed that in DTA's view, the original compliance letters were confusing and complex. He observed that DHS has since moved away from the old waterfall testing model, adopting the Digital Service Standard.[101]

6.94      Noting that DHS had done a lot of user testing, Mr McNamara commented that it is the nature of how the user testing interacts with the design. He stated that DHS has evolved and has become a lot more interactive with the customer early on in the design stage. He noted that the Digital Service Standard is more about an interactive way of doing things.[102] Mr Alexander noted that the Digital Service Standard would 'not solve all the problems of the world'; it is a set of 13 processes that, if followed, solves a lot of problems. There is now a government-mandated standard for agencies.[103]

6.95      Dr Seebeck observed that until a system is 'tested in the wild' you are not going to have a full understanding of it, and that this goes back to understanding the test data.[104] She said the OCI was a good example of automation; that what DHS did with the testing was done with the best of intent:

It is putting it in the wild, learning to go through the beta and coming back and speaking to us [DTA] about the user-centred design.[105]

Committee view

6.96      The "Robo-debt" project applied many of the techniques of digital transformation with none of the underlying principles. Data matching and automated decision making could, in other contexts and with appropriate safeguards, make positive contributions to the delivery of government services. This project, however, represented a failure in policy conception and design.

6.97      The committee finds it extraordinary and disturbing that the department could describe the project has having gone 'very well' despite the well documented hardship and distress it caused countless Australians.

6.98      That evaluation was a direct result of the very limited conception of digital brought to the project by both the department and the minister.

6.99      Where digital transformation is undertaken solely to reduce costs or identify savings, it is likely to do so at the expense of user experience.

Welfare Payment Infrastructure Transformation[106]

Background

6.100         In 2014, the Treasurer, the Hon. Joe Hockey MP said that the government had no choice but to replace Centrelink's information and communication technology system arguing that problems affecting the system were affecting the quality of services to customers.[107]

6.101         As part of DHS's Digital Transformation Strategy, the Welfare Payment Infrastructure Transformation Programme (WPIT) was implemented to replace the ageing Centrelink ICT system.[108] DHS described the project as a business led, user centred, technology enabled project that will be delivered in five stages over seven years from July 2015–2022. 

6.102         The project is intended to:

  • provide customers with faster, more connected and automated digital services;
  • give staff a modern ICT platform that makes it easier for them to do their jobs;
  • position the department to meet future policy needs of government.[109]

6.103         The 2015–2016 Budget allocated funding of $60.5 million over four years from 2015–2019 to deliver the first stage of WPIT—Tranche one—by 31 December 2016. 

6.104         Tranche one covered:

  • Business planning, scoping and design and the implementation of digital improvements and new services; the selection of the major commercial partners for the project;
  • Digital enhancements:
  • a 'claim tracker' capacity to allow welfare recipients to track the progress of their claims;
  • an online 'payment and service finder' to help potential claimants to understand what payments and services might best suit their circumstances;
  • the introduction of virtual assistant, 'Sam', on the families and students website pages;
  • a new training tool to assist staff in their jobs; and
  • a new Scaled Agile Framework (SAFe) delivery model, which has assisted in delivering in ways of working to implement incremental changes for both staff and students.[110]

6.105         According to the DHS 2016–17 Annual Report, the WPIT budget measures initiatives for Tranche one were delivered as agreed and within timeframes and on budget.[111]

6.106         In December 2016, the government committed $313.5 million net expenditure over four years for the delivery of the second stage—Tranche two. This tranche marked the transition from the foundation planning and set up phase into the first of its core delivery phase. The tranche is to focus on student payments for Youth Allowance and Austudy, with some student claim processing being automated, and is expected that decisions on student's payment applications will be delivered more quickly.[112] 

6.107         Together with DHS's commercial partners, tranche two will also address the co-designing of the core system features required to support all welfare payments into the future—that is, to provide a template for the delivery of payments for job seekers, families, older Australians and people with disability.

6.108         During tranche two a concept of operations is also to be developed for the delivery of tranches three to five.

6.109         On 22 March 2017, the Minister, the Hon Alan Tudge, announced that Accenture had been selected as the preferred tenderer to provide systems integration services for the delivery of welfare payments.[113] An article in Computerworld reported that Accenture will work with Capgemini, IBM, and HP Enterprise as members of a panel that will provide systems integration services. Computerworld reported that SAP is the preferred software vendor for the WPIT.[114]

DHS Response

6.110         Mr John Murphy, Deputy Secretary, Department of Human Services advised that a budget measure in the 2013–2014 financial year funded a two-year study to look at options to replace ISIS. This preliminary work was funded by DHS. WPIT was announced as a budget measure in the 2015–2016. Mr Murphy explained that DHS is currently operating a system that was largely designed in the 1970s, 1980s and 1990s based on paper, telephones and face-to-face interactions. At its heart, the WPIT project is designed to transform DHS's businesses processes:

...from my point of view,...this really is about a business transformation that needs to really stare into 30 or 40 years of complexity that has been built up. As I said earlier, I have come from the private sector. One of my observations, having joined government, is that this is a far more complex environment than the one I'm used to working in. My banking colleagues may not like me for saying that, but it is more complex because of the nature of the services we provide. Therefore, what we need to do around digital is more challenging. Also, the way that we think about supporting our customers, our citizens, needs to be far more layered and for more nuanced than you would see in the private sector.[115]

6.111         Mr Murphy noted the challenges facing WPIT to be:

  • the redesign of the business processes;
  • to change the culture by the reskilling, the retraining of our people; and,
  • the right-sizing of the department, by having the right people in the right place; and then the technology.

6.112         Mr Murphy told the committee that the expectations of customers have increased significantly, including doing things digitally, and to be able to access services at a time of their choosing, noting that the idea of normal business hours fell away many years ago.[116]

6.113         WPIT is being approved and progressed in staged tranches. Work on the project is undertaken only to the extent of the government approved level of funding.  In the case of the WPIT program, work is approved to 30 June 2018.[117] Mr Murphy observed that from his experience, the level of investment in WPIT is not unusual, particularly given the nature of the transformation of the WPIT program, which is to truly transform the welfare part of DHS, which is a significant undertaking.[118] He stated that he was confident that the value that is being delivered is matched by the level of investment. He noted that $104 million was returned to the budget in the last MYEFO as it was not required. [119]

6.114         The second tranche of WPIT, which commenced on 1 January 2017, and concluded on 30 January 2018, focussed on students who access Youth Allowance and Austudy. The reason for choosing the student category was because as a group students are more digitally literate, so are a good group for testing systems design. The funding for tranche two was $313.5 million.[120]

6.115         Mr McHardie advised that DHS is not focussing on one core technology end to end: that the process 'is not a monolithic ICT build'. WPIT is replacing the monolithic ISIS system built in the 1980s by one vendor. The new system is using the right technology where it fits best across the four core functional blocks: interaction with the welfare recipient; claims processing; eligibility assessment; and a payment figure. All customer-facing screens are being built in-house by DHS, using open source software called Angular, which stops vendor lock-in. The eligibility assessment is built around the SAP business suite, where DHS has taken a vendor non-industry specific platform and custom developed it into DHS's own industry specific platform. A SAP payments system currently in use in the department to pay its own employees and several other departments is being used for tracking of the payments that go out to Australian citizens.[121]

6.116         Mr McHardie noted that DHS has gained considerable experience within the department of doing custom development work on the core SAP platform. Almost 500 public servants are now qualified and certified as SAP professionals, whether they are enterprise architects, developers or testers:

We now have a lot more control of our destiny, particularly when we need to work on core products such as SAP.[122]

6.117         Mr McHardie advised that the cost profile of WPIT has not changed. In having a better understanding of the available products and a capacity to do work in-house, DHS is in a better position to cost projects than was previously the case. DHS is constructing the WPIT program on the basis that each tranche is largely self-contained. The benefits are derived from the tranche, rather than being reliant on the final piece of work being delivered:

The days of monolithic 10-year programs, when you wait to the end to get the value, and I mean value both in terms of customer experience and also in terms of financial benefit, that boat has well and truly sailed.[123]

6.118         Mr Murphy observed that from a business point of view, it's not whether it's a large vendor or a series of small vendors; it's their ability to respond to the business need. It is not a question of big or large. What is needed is technology that is able to respond to the business need. I'm expecting that, in a program of the scale of the WPIT program a mix of capabilities will be brought to bear. DHS does not want to lock itself into a technology that might be overtaken in two, three, four, five years, for example.[124] Mr McHardie advised that DHS carries the risk on the in-house approach to the delivery of the ICT, whereas if a systems integrator had been engaged to build the system from top to bottom, the systems integrator would bear the risk.[125]

6.119         Mr Murphy further advised that DHS has adopted a key-performance-indicator (KPI) framework for its contracts with commercial partners. He stated that it is an evolving framework, but a considerable amount of discussion with commercial partners is about risk. Within the WPIT project, there has been a shift from a procurement focus where DHS chose ITS major commercial partners, to now managing the commercial partners using KPIs, frameworks, discussions and course setting.[126]

Committee view

6.120         The replacement of Centrelink’s legacy system, ISIS, represents a mammoth undertaking. The approach DHS has taken appears to represent a substantial divergence from the patterns of the path. It is too early to tell, but the committee is hopeful that this decision may pay dividends.

6.121         The committee is heartened by two aspects in particular.

6.122         First, DHS has acted to build its internal ICT and digital expertise. Almost 500 public servants are now qualified and certified as SAP professionals, whether they are enterprise architects, developers or testers:

We now have a lot more control of our destiny, particularly when we need to work on core products such as SAP.[127]

6.123         This has not substantially increased the cost profile:

CHAIR: If you were to compare the input costs for projects back in 2013, when you were more reliant on external providers to deliver these interactions with these big platforms, and where you are now, where you've got 500 people who are accredited and on staff, does that produce a different set of cost drivers when you're scoping up a project for the future?

Mr McHardie: I think it does in the initial costings that are put together for projects, particularly when there are government directed activities, where government is looking at a range of solutions that it could roll out to meet legislative change or new legislative policy, or when replacing large elderly legacy systems. We understand these products so much better now, and with us doing the in-house build we're able to cost up those bodies of work much more effectively.

CHAIR: So you're a more informed buyer when you do go externally, but you're also able to deploy internal labour to drive down cost?

Mr McHardie: Correct.[128]

6.124         The committee believes that this is a clear demonstration of the benefits of bringing expertise in house.

6.125         Second, DHS has moved to shift more risk onto its contractual partners:

The DHS has also raised the spectre of shared incentives to ensure it gets the best possible resources for its vendors. It is yet to make a final decision on specifics, but flagged an increase in base fees as a reward for excellent performance scores, balanced by reduced base fees for low performance.

"Under each work order it is likely that fees will only be paid if specified miles are met" the department warned.

"There may also be liquidated damages payable for late delivery, and other remedies available to the department for poor performance."

..."The contractual arrangements are likely to require SI panel members to accept significant financial risk," it warned. The department is only inviting bids from organisations that have the depth to commit "significant financial and human resources" over the full five to seven years the project is estimated to take.

It cautioned that contractors will only be paid once milestones are successfully passed—meaning systems integrators could have to fund a significant commitment of resources for some time before they receive their first dollar.

Key executives from the successful panellists will also be expected to commit facetime to the department, especially if any disputes arise.

DHS has made it clear that panellists will not be paid for any work they put into the "competitive dialogue" phase that will result in integrators being picked for each different tranche of work.[129]

6.126         The committee is hopeful that building in house expertise and making contractual partners take responsibility for delivering to contract will help prevent some of the appalling waste and delay that has marred previous projects. It will remain to be seen whether the government is able to execute this new approach in the roll out of the remaining tranches of WPIT.

Senator Jenny McAllister
Chair

Navigation: Previous Page | Contents | Next Page