Chapter 4

The ATSB's accident investigation processes

Overview of the investigation process

4.1        The safety investigation process is geared towards improving safety by shedding light on factors which led to a given occurrence and making safety-related recommendations, thus reducing future risk.

4.2        Following an occurrence, investigations are conducted in order to improve safety by determining what confluence of events or factors led to the event. As put by the Australian Transport Safety Bureau (ATSB):

The purpose of a safety investigation is to enhance safety, not to apportion blame or liability.[1]

4.3        An occurrence is defined as an accident or incident. Standard terminology used to refer to key safety and risk concepts includes but is not limited to:

• Safety factor: an event or condition that increases safety risk.
• Contributing safety factor: a safety factor without which the occurrence would not have happened.
• Other safety factor: a safety factor that, while not meeting the above definition, nonetheless is important enough to warrant inclusion in an investigation report.
• Safety issue: a factor that has the potential to compromise the safety of future operations, or is characteristic of a system or operational environment.
• Risk level: the level of risk associated with a particular safety issue.[2]

4.4        The ATSB advised the committee that, due to the emphasis it places on future safety, the agency employs what it calls a 'link-by-link' approach during its safety factor analysis and investigations. This means that judgements about whether a particular safety factor contributed to an occurrence are made in terms of the factor's relationship to another contributing safety factor. Other types of investigations, the ATSB posited—particularly those aiming to determine responsibility for an event—generally employ what is called a 'relative-to-occurrence' approach, whereby judgements about the extent to which a factor contributed are made in terms of the factor's direct relationship to the occurrence in question. The ATSB contended that its approach sets it apart from other proceedings:

The ATSB analysis framework involves a higher standard of proof than in Australian coronial inquests or civil legal proceedings or factors relatively close in proximity to the occurrence (that is, more than 66 per cent versus more than 50 per cent).[3]

4.5        Based on the ATSB's submission, the committee understands that the agency follows a fairly prescriptive investigative approach. As put by the ATSB, the process applied in each instance follows defined procedures and protocols:

The ATSB Safety Investigation Quality System (SIQS) provides policy, procedures, guidelines and tools for the conduct of all key investigation activities.[4]

4.6        These procedures and protocols are 'designed to ensure consistency in methodology and implementation of the provisions of the TSI Act'.[5] Broadly speaking, the processes consist of notification and assessment of the accident or incident in question, investigation, analysis and reporting.[6] The ATSB provided the following figure depicting a high level view of the principal processes involved.

Figure 4—ATSB investigation process

Investigation and analysis

4.7        The investigation aspect of the process above involves an initial response following an occurrence, the initiation of an investigation and data collection. The ATSB advised that its investigation processes 'may or may not involve an on-site visit'.[7]

4.8        Collected data typically involves coverage of a wide range of topics applying different techniques. Data can be physical, testimonial, documentary or recorded. Information and evidence is gathered on the sequence of events, the personnel, organisations and equipment involved and environmental factors, but may also include a wider range of material as deemed necessary.[8]

4.9        Following the information gathering stage the process moves to the analysis stage, where data are reviewed and converted into a series of arguments, or excluded, if the ATSB assesses via its risk matrices that the future risk to high capacity operations is low.[9] These in turn produce a series of conclusions, which are primarily concerned with safety issues and contributing factors. The ATSB advised the committee that 'analysis relies on informed judgement and is, to some extent, subjective'.[10]

4.10      Once a draft report is prepared by the investigator-in-charge and the investigation team, it is subject to a review and approval process. This process comprises several stages:

• internal, peer and management reviews of the draft report;
• approval of the draft report for release to directly involved parties (DIPs);
• assessment of DIP comments by the investigation team and/or the investigator-in-charge;
• finalisation of the final report;
• review and approval of the final report by the ATSB commissioners;
• advance release of the final report version to DIPs and other relevant parties; and
• finally, public release of the report.[11]

DIP process

4.11      DIPs are individuals or organisations that were directly involved in an occurrence or may have influenced the circumstances involved in its creation. They may also include individuals or organisations whose reputations could be affected following the public release of an ATSB investigation report. These typically include the regulatory authority, in this case the Civil Aviation Authority (CASA), the crew and the operating organisations, in this case Pel-Air.[12]

4.12      The DIP process provides these individuals and organisations an opportunity to make submissions on the factual accuracy of an investigation report prior to its public release. Reports are distributed to DIPs according to the matrix below:[13]

Figure 5—Advanced release of ATSB reports

4.13      Should DIPs believe that an investigation report contains factual inaccuracies or omissions, they may provide evidence in support of this view. Their submissions and evidence are assessed and the information is either ‘noted’, ‘accepted’, ‘partly accepted’ or ‘rejected’ along with a written justification for the assessment. The ATSB may decide that no further action is required, or that the information warrants further investigation or that changes be made to the final report.[14]

4.14      This process, the ATSB advised the committee, provides an opportunity for natural justice to these parties.[15]

4.15      The committee understands, however, that the Transport Safety Investigation Act 2003 (TSI Act) does not provide for transparency in the DIP process, and questions how ATSB decisions regarding the inclusion or omission of DIP information can be assessed.

4.16      The committee notes that other jurisdictions, such as the United States, apply a higher degree of transparency in this regard. The committee recommends a course of action later in this chapter to ensure that DIPs have access to a fair and valid process, and that appropriate checks and balances are in place.

ATSB report approval and release

4.17      Once the DIP process has been finalised, the report is reviewed by the Manager and General Manager before being approved for publication. Under section 25 of the TSI Act, this approval can only be given by the Commission and cannot be delegated.[16]  

4.18      Should new information come to light following the public release of the final investigation report, the ATSB advised the committee that its policy 'provides for the reactivation of any transport safety investigation in circumstances where new and significant information (in relation to the matter that was investigated) is brought to the attention of the ATSB.'[17]

4.19      The committee noted that relevant new information may include information presented during the course of a coronial inquiry that was not previously made available to the ATSB, new physical evidence, or the results of research which may be directly relevant.[18]

The ATSB risk matrix

4.20      As previously stated, the term 'risk level' refers to the risk ascribed by the ATSB to a particular safety issue. Under the ATSB's classification system there are three categories of safety issue:

• Critical safety issue: associated with an intolerable level of risk.
• Significant safety issue: associated with a level of risk that is acceptable if kept as low as reasonably practicable.
• Minor safety issue: associated with a level of risk that is broadly acceptable.[19]

4.21      Risk levels are noted in the 'Findings' section of ATSB investigation reports.[20]

4.22      The ATSB advised the committee that its risk analysis process was consistent with the Australian and international standard,[21] and summed up its methodology in the following manner:

The ATSB risk methodology examines the worst credible occurrence scenario in terms of its likelihood and consequence to establish the safety risk associated with the identified safety issue. Likelihood and consequence tables are used to inform this assessment. Application of the worst credible scenario accounts for the effect of in-place risk controls and management processes that generally act to reduce the level of adverse consequences associated with the worst possible scenario.[22]

4.23      The classification of safety issues as 'critical', 'significant' or 'minor' determines the effort which the ATSB will apply towards facilitating safety action. The process is depicted by the figure below:[23]

Figure 6—ATSB risk analysis process

 

4.24      Currently, any prospective aspect of the ATSB risk assessment process will only have validity if the agency is correct in its highly subjective analysis, as there is no opportunity for others to make that same assessment. Explaining what went wrong in the instance under consideration allows the whole industry to assess which lessons may apply to future operations.

4.25      The committee notes that assessing risk is not an exact science, but is rather an attempt to predict the likelihood and possible consequences of an event occurring on the basis of limited data of uncertain quality. Risk assessments are therefore often by necessity conservative.[24]

4.26      In this vein, the ATSB advised the committee that safety investigations assess situations as they existed at the time of an accident, considering the risk involved as it relates to one of two possible scenarios:

• Worst possible scenario: the worst and most severe occurrence that could eventuate as a result of a safety issue.
• Worst credible scenario: the worst and most severe occurrence that could eventuate as a result of a safety issue, determined after consideration has been given to the risk controls and management processes in place.[25]

4.27      To explain how the agency employs the above scenarios in estimating consequences and likelihood levels, the ATSB submitted:

Using the worst possible scenario as the basis of estimates of consequence and likelihood levels will generally lead to the selection of the highest level of consequence in the risk matrix. It is technically possible that almost any safety issue could result in a catastrophe. Even in the worst credible scenario, regard needs to be given to the normal expectation of compliance with existing risk controls, such as rules and standard operating procedures.[26]

4.28      A table[27] describing how the ATSB decides the scale of consequence ratings during safety risk assessments was also provided:

Figure 7—ATSB scale of consequence

4.29      Likelihood is rated into one of four categories:

• Frequent – expected to occur at least once per year.
• Occasional – would probably occur in the medium term, approximately once per decade.
• Rare – could occur in certain circumstances, possibly once per 100 years.
• Very rare – would only occur in exceptional circumstances, possibly once per 1000 years.[28]

4.30      Scales of consequence and likelihood are then used to inform the risk rating matrix. The matrix[29] takes the following form:

Figure 8—ATSB risk rating matrix

Risk analysis and the Norfolk Island accident

4.31      In the context of the VH-NGA accident, the ATSB advised the committee that the worst credible scenarios examined were 'significantly influenced by the in-place risk controls and management processes.'[30] These controls and processes included the requirements and guidance set out in Pel-Air's operations manual.

4.32      Essentially this means that, adhering to what appears to be a fairly rigid process of risk assessment, the ATSB chose to work on the assumption that proper risk controls, such as adequate operating procedures and their oversight, were in place.[31]

4.33      Looking at the ATSB's consequence table (Figure 7), the committee observes that according to the ATSB's assessment processes, even if all six people on board VH‑NGA had died, the highest possible consequence attributable to the accident would have been 'moderate'.

4.34      The committee put this proposition to the ATSB. Chief Commissioner Dolan responded:

That is a simplification of the purpose of that table. We will do a risk assessment of an identified safety factor. This is not about assessment of evidence, this is about assessment of safety issues—a safety factor that is seen to have a continuing effect on risk to assess the likelihood and the consequence of that factor coming into play in the future. That is our basis for establishing the significance of a safety issue. It is not the basis on which we will assess evidence.

If you are looking for the philosophical underpinnings of how we deal with evidence and a range of other things, there is a document, Analysis, causality and proof in safety investigations, which was a publication of Dr Walker and Mr Bills in 2008. That shows the philosophical underpinnings of how we deal with facts, evidence, analysis and so on. It is reflected in our policies and procedures in the organisation. The risk assessments largely draw on or are compressed versions of international safety organisation risk management standards. We are trying to bring all that to bear on a diverse range of operations, while bearing in mind the guidance from the government that our attention should primarily be on the safety of the travelling public.[32]

Committee view

4.35      The ATSB's response notwithstanding, the committee remains concerned by the fact that the highest consequence the ATSB would attribute to the safety issues for those involved with emergency medical flights—in this particular case the patient, her family, the medical staff and flight crew—is 'moderate'. This would be the case even if all six on board had died in the accident.

4.36      The committee is highly sceptical of a risk analysis process which can produce such a result. If the application of this methodology continues, the systemic and oversight deficiencies which allowed the VH-NGA pilot-in-command to be the last line of defence would remain unchanged.

4.37      It is important to note that current regulations include 'ambulance functions' under the category of 'aerial work', as outlined in Civil Aviation Regulation (CAR) 206. CAR 206 sets out what is referred to as the "classification of operations" and establishes three broad classes of commercial aviation: aerial work, charter and regular public transport (RPT).  Those classes of operations reflect two things: the exposure of the general public to the inherent risks; and the presumed knowledge of and acceptance of risk by the participants.  The class specific regulatory requirements, set out in Part 82 of the Civil Aviation Orders (CAOs), become more demanding as more people are at risk and as they have less control over individual outcomes.  Aerial work has the lowest compliance requirements and RPT has the highest.

4.38      The ATSB has codified the allowable thoughts and actions of its investigators to the extent that common sense and intuition appear to be extinguished. This codification appears to be based on a false premise that the ATSB can correctly predict future risk and is in fact the only organisation that can or should have access to the facts of an incident where such application to other current or future operations is made. The ATSB is so far removed from the many and varied operations of Australia's Air Operator's Certificate (AOC) holders that such a premise is demonstrably flawed.

4.39      It is therefore imperative that the ATSB mitigate the unintended consequences of the interaction between its risk assessment processes and the strategic guidance from the minister.[33] Separately, the minister should review the strategic guidance to ensure it does not elicit these unintended consequences.

4.40      The committee considers the ATSB's approach to this investigation a lost opportunity for industry to learn. Given that the minister's current Statement of Expectations is valid until 30 June 2013, the committee considers this a good opportunity for mitigating any unintended consequences.

Recommendation 2

4.41      The committee recommends that the minister, in issuing a new Statement of Expectations to the ATSB, valid from 1 July 2013, make it clear that safety in aviation operations involving passengers (fare paying or those with no control over the flight they are on, e.g. air ambulance) is to be accorded equal priority irrespective of flight classification.

4.42      The committee also believes that ATSB should move away from its current approach of trying to forecast the probability of future events and conduct thorough examinations of the reasons for accidents. This would allow the industry to make its own assessment of the factors and their relevance to their own operations.

Recommendation 3

4.43      The committee recommends that the ATSB move away from its current approach of forecasting the probability of future events and focus on the analysis of factors which allowed the accident under investigation to occur. This would enable the industry to identify, assess and implement lessons relevant to their own operations.

Downgrading of the critical safety issue

4.44      The committee learned of the existence of a critical safety issue identified early on by the ATSB, and considered how the safety issue and risk assessment processes outlined above may have watered down the outcome of the ATSB's investigation into the VH-NGA accident.

4.45      The committee expended considerable effort in trying to piece together how and why the downgrading happened. A synopsis is offered below.

4.46      Initially classified as a critical safety issue, the final ATSB report identified the following minor safety issue:

The available guidance on fuel planning and on seeking and applying en route weather updates was too general and increased the risk of inconsistent in-flight fuel management and decisions to divert.[34]

4.47      With the information available to them once they became aware of the deteriorating weather conditions, the crew perceived that diversion carried a greater risk than continuing to Norfolk Island.[35] The committee understands that regulatory and guidance material did not require the crew to divert in the particular circumstances they faced.[36]

4.48      CASA conceded this point, but was nonetheless of the view that a diversion should have occurred:

CASA’s position with respect to the diversion issue was and remains that, in all the circumstances of the accident flight, good airmanship should have resulted in a diversion, even if there was no explicit, mandatory requirement that the accident pilot do so.[37]

The ATSB position

4.49      Early on in its investigation, the ATSB formed the provisional view that the inadequate en route guidance was a key issue, and drew this to CASA's attention:

Because we take a prudent view of these things, we very strongly stated what we saw as provisionally the risk that was involved here—the risk that the guidance about en route management of these flights was not adequate.[38]

4.50      The ATSB examined the guidance available to the flight crew in the Aeronautical Knowledge Syllabus (ATPL(A)), asked a group of 50 ATPL students what they would do under similar circumstances, examined a number of operations manuals from similar operators and interviewed a sample of pilots. They did not find consistent knowledge or processes.[39]

4.51      Evidence indicates that the ATSB was right to highlight this lack of guidance; it was something that pilots and CASA's Approved Testing Officers had grappled with for years.[40] It was also posited that clearer guidance might have helped avoid the VH‑NGA accident.[41]

CASA's response

4.52      The committee understands that CASA officers responded positively to the ATSB's initial assessment of the issue in meetings held at the officer level in February 2010.[42]

4.53      What transpired in discussions at the CASA senior management level is not known to the committee. The committee does know however, that concerns were raised within CASA about the possible ramifications of the identification of the critical safety issue on CASA's legal actions against the pilot in command.[43] Nevertheless, in its formal written response to the ATSB, CASA indicated that the current legislative regime and aeronautical knowledge training requirements were, in its view, sufficient to ensure that pilots make appropriate in-flight decisions.[44]

4.54      CASA's Director of Aviation Safety, Mr John McCormick, denied that his agency had at any point agreed with the ATSB's assessment of the safety issue as 'critical'.[45] It was, however, his understanding that the ATSB initially planned to make a recommendation on the issue.[46]

Information withheld from the ATSB

4.55      In informing the ATSB that it did not consider the identified safety issue 'critical', CASA did not communicate the results of an internal survey of its Flying Operations Inspectors (FOIs).[47] The six[48] FOIs, all pilots, were asked what they would have done in a similar situation. The result was anything but conclusive:

Our FOI population seem to be evenly split about the need, nor not, to mandatorily divert to an alternate from the last point of possible diversion if the destination weather falls below alternate minima. Indeed the material prepared to go to the AAT [Administrative Appeals Tribunal] in response to the James [VH-NGA pilot in command] matter currently makes the statement that, because the weather at Norfolk had fallen below alternate minima, a diversion at or before the latest diversion point was mandatory. The basis for this split seems to be a statement in the AIP [Aeronautical Information Package] suggesting that this is a 'legal' requirement. The other half believe that this is not the case and that the aircraft commander could continue to destination, even of the aircraft was not carrying alternate fuel. This is a position we must settle definitively, along with a number of other planning and in-flight decision making issues.[49]

4.56      Another senior CASA officer concluded that this could reflect badly on CASA:

...[T]here is one group of pilots that have one view which leads to a mandatory diversion and another group with the opposite view. Putting aside the practicalities, both groups believe they are legally correct. If we find ourselves in an AAT, or a court we once again look a bit foolish if we, the regulator, find ourselves in a position were [sic] we have to say there are two conflicting views, one of which has to be wrong, and we have done nothing to rectify that over the years. Very untidy.[50]

4.57      Irrespective of the absence of consensus among CASA's own pilots on what they would do in circumstances like those faced by the VH-NGA crew, Mr McCormick held firm in his view that VH-NGA's crew should have diverted.[51] When discussing the FOI split he did, however, conceded that 'there is work to be done in that area,'[52] but also rejected the proposition that clear guidance could be written.[53]

4.58      Evidence provided to the committee argued that other jurisdictions, for example in Europe[54] and Hong Kong, have, unlike CASA, been able to draft relevant guidance.[55]

4.59      The committee heard that proposed Civil Aviation Safety Regulation (CASR) Part 135 may assist in dealing with this issue.[56] It was argued that proposed Part 135 is a positive development but this guidance was not available at the time of the accident and therefore it was rightly categorised by the ATSB initially as having a critical effect on safety.[57]

How the issue was downgraded

4.60      The ATSB advised the committee that having a) considered CASA's position when downgrading the safety issue, and b) conducted its own risk assessment (following processes outlined earlier in this chapter), the agency's view of the criticality of the safety issue changed.[58] In effect, the ATSB subsequently satisfied itself that sufficient guidelines were in place, and the risk level attributed to this type of flight did not warrant deeper investigation.[59]

4.61      Chief Commissioner Dolan explained that the risk assessment—likelihood and consequence—and report review process led to the issue being downgraded:

What we tend to do is have a series of review steps—peer review, managerial review, and finally review by myself and my fellow commissioners, of reports at various stages...on the way through, as we checked and reviewed the position according to our methodology, progressively we were less convinced in our framework that this was as significant an issue as we first thought.[60]

4.62      The agency's initial categorisation of the issue as 'critical' was, Mr Dolan explained, the result of preliminary fact-gathering and erring on the side of caution.[61]

Did CASA and the ATSB collude?

4.63      Documentation made available to the committee raises questions about the level of influence CASA may have had during the ATSB investigation.[62] It is clear that the ATSB Chief Commissioner, Mr Martin Dolan, knew that CASA did not support a broad systems approach to the inquiry despite earlier indications to the contrary from Mr John McCormick, head of CASA.[63] Furthermore, early in the investigation there appears to have been cross checking of the CASA investigation report with the ATSB draft to ensure they were consistent.[64] In addition, at least one high level meeting was supposed to have occurred between the two agencies on the safety issue but was not minuted.[65] The committee is also aware that both the ATSB's General Manager, Mr Ian Sangston, and Chief Commissioner Dolan personally reviewed the report draft. ATSB documents provided to the committee indicate that an evidence table was reworked in order to reflect Mr Sangston's final assessments that the identified safety matters were 'minor safety issues'.[66]

Committee view

4.64      The committee can draw no firm conclusions regarding allegations of collusion, as high level meetings and review processes were not minuted or documented. The committee notes, however, that the safety issue was downgraded after Mr Sangston's meeting with CASA. While the committee cannot be conclusive, in the absence of more transparency from the agencies concerned the committee appreciates the unease voiced as to the motivations behind changes made to the report.

4.65      In the committee's view there is sufficient evidence to conclude that the ATSB's 'consequence and likelihood' risk assessment process had the effect of trivialising the risk posed by inadequate guidance available to the flight crew. This was the matrix used to downgrade the safety issue from 'critical' to 'minor'.

4.66      The committee notes support for the ATSB's initial categorisation of the issue as 'critical'. The committee also notes the view of Mr McCormick against prescriptive guidance as an effective risk control is surprising given his position as CASA's Director of Aviation Safety. Mr McCormick appears to lack confidence that his organisation can write regulations and guidance material that is simple, clear and unambiguous.

4.67      The committee does not intend to second-guess CASA on technical detail as to whether pilots should divert, but notes evidence indicating that clearer guidelines can, and have been, drafted by other overseas aviation safety agencies. What is incontestable however, is that pilots are divided in their reading and understanding of the current guidelines,[67] and the question of whether guidelines were adequate is certainly not black and white. CASA's decision to withhold this fact during discussions with the ATSB, and to instead offer assurances that the guidelines were sufficient, could be seen as a misrepresentation of reality. It certainly affected the severity and scope of the identified safety issue.

4.68      The committee is concerned by the fact that no paper trail exists clearly documenting the ATSB's decision to downgrade the issue. Should a similar accident occur in future, this fact will surely be seen as a missed opportunity to enhance safety. The reasoning behind the downgrade, and the process and evidence leading to it, appears at the least unclear.

Recommendation 4

4.69      The committee recommends that the ATSB be required to document investigative avenues that were explored and then discarded, providing detailed explanations as to why.

The way forward

4.70      Given the suboptimal effect of the ATSB's rigid and subjective processes on the VH-NGA investigation report, the committee considered a number of ways to encourage improvements in the conduct of safety investigations and production of reports. These revolve around the remit of the agency, the expertise of its leaders and quality control of its product.

Effect of change from BASI to ATSB

4.71      The ATSB was formed in 1999 following the amalgamation of the Bureau of Air Safety Investigation (BASI), the non-regulatory parts of the Federal Office of Road Safety (FORS) and the Marine Incident Investigation Unit (MIIU). Prior to this amalgamation, the air safety investigator, BASI, focused exclusively on aviation transport.

4.72      Given that BASI was specifically tasked to investigate aviation accidents, whereas the ATSB has a much broader modal remit, the committee sought views on the effect of the amalgamation of three separate agencies into one.

4.73      The committee heard that this approach was not standard practice internationally. Investigation agencies in the United States (NTSB), the United Kingdom (AAIB) and New Zealand (TAIC) all have aviation accident investigation as their primary, and in one case only, function. Whilst these agencies cover modes of transport other than aviation, the difference appears to be that they have retained the higher standards of the aviation accident investigations community rather than allowing standards to decrease toward the other modes. All of these agencies are also tasked with determining causes of accidents, whereas the TSI Act tasks the ATSB with identifying factors which contribute to transport safety matters.[68]

4.74      The committee was informed that BASI had pioneered a high standard of work internationally in its time.[69] Up until the mid-1990s, the agency also had a sound depth of experience in technical aviation-related matters. This began to suffer as a consequence of regional office closures and the ensuing loss of highly experienced investigators.[70]

Training

4.75      The competence and training of accident investigators working for the ATSB are also of concern.

4.76      Australia has a very limited aircraft construction industry, and has for a long time struggled to retain technical investigators with a depth of experience with large aircraft operations.[71] 

4.77      To address these shortcomings, the committee was told that the theoretical internal investigator courses the ATSB conducts simply cannot replace technical experience, and should be supplemented with training offered by the NTSB and AAIB.[72] The committee supports this view.

Recommendation 5

4.78      The committee recommends that the training offered by the ATSB across all investigator skills sets be benchmarked against other agencies by an independent body by, for example, inviting the NTSB or commissioning an industry body to conduct such a benchmarking exercise.

Recommendation 6

4.79      The committee recommends that, as far as available resources allow, ATSB investigators be given access to training provided by the agency's international counterparts. Where this does not occur, resultant gaps in training/competence must be advised to the minister and the Parliament.

Expertise of Commissioners

4.80      The committee is of the view that the quality of the ATSB's work in aviation safety is significantly tied to the expertise of its commissioners. The importance of such expertise is highlighted when commissioners are reviewing investigation reports.

4.81      At present, the ATSB is not being led by individuals with a high degree of aviation expertise, which could, in part at least, explain the questionable quality of the Norfolk Island report.

4.82      For his part, Chief Commissioner Martin Dolan has worked as a Commonwealth public servant for 30 years. His aviation experience prior to joining the ATSB is limited to his 2001–2005 role as Executive Director of the Aviation and Airports section of the Department of Transport and Regional Services, where he had responsibility for airport sales and regulation, aviation security, aviation safety policy and international aviation negotiations.[73] It is unlikely that this role would have furnished him with significant technical knowledge, aviation operational or investigative experience. In turn this means that the value added by report reviews is limited to ensuring that the process had been followed, rather than providing any insight into whether the result of the process is logical.

4.83      The other two Commissioners, Ms Carolyn Walsh and Mr Noel Hart, have no aviation experience. They do however have experience in the other two areas of the ATSB's remit.

Committee view

4.84      The committee notes BASI's strong reputation for aviation investigation expertise, and that this reputation might have begun its current period of decline in the years since BASI was amalgamated with two other agencies. The committee has to ask whether the amalgamation had the unintended consequence of eroding BASI standards and expertise down to the relatively lower level of FORS and MIIU.

4.85      Although the committee accepts that the move from BASI to the ATSB was well-intentioned, the possible impact on the time and proportion of resources that can now be attributed solely to aviation safety matters is concerning. With aviation now being just a part of a larger organisation led by people without world's best practice expertise in aviation systems safety, it should come as no surprise that, over time and without an informed leader advocating for adequate resources or focus, ATSB standards have fallen short of international peer organisations. The committee is not suggesting that a separate agency with responsibility for aviation safety investigations should be established, but is of the view that improvements could be made which would bolster the ATSB's aviation credentials.

4.86      To this end, the committee is of the view that knowledge has to start at the top. This requires commissioners to have the appropriate competence, in terms of both qualifications and experience, in safety management systems, which, the committee notes, is predominantly found in individuals with expertise in aviation and petrochemical fields. Furthermore, the lack of aviation expertise in the upper echelons of the ATSB would certainly appear to be directly in conflict with annual report statistics which suggest that 80 per cent of all investigations instigated by the ATSB are aviation related.[74] The committee therefore has to conclude that the current Chief Commissioner's aviation safety experience is not adequate for the task at hand.

Recommendation 7

4.87             The committee recommends that the Transport Safety Investigation Act 2003 be amended to require that the Chief Commissioner of the ATSB be able to demonstrate extensive aviation safety expertise and experience as a prerequisite for the selection process.

Industry experience and risk-based aviation support

4.88      The committee was deeply concerned by the consequences of the ATSB's rigid risk assessment processes, noting specifically the adverse effect these processes had on the Norfolk Island investigation.

4.89      As previously outlined, the risk matrix the agency employs looks at the consequence of an accident, and only accidents involving large aircraft carrying fare‑paying passengers can reach the highest consequence level. This is something the committee rejects. There is no excuse for lapses in regulatory oversight, and the ATSB should be obliged to investigate fully any accident with passengers involved—passengers who have reason to believe that they are being transported by a professional organisation (whether that be an airline, charter operator, rescue helicopter, flying doctor service or international rescue service paid for by their travel insurance). There is no sense in partial investigations or patchy surveillance.

4.90      Air Operator's Certificate (AOC) holders who conduct a range of operations which include non-standard mission profiles and routes, often flown at short notice, require a greater degree of review and regulation than they appear to be subject to at present. A shift in this direction is an option the committee believes should be explored, and one which would be in line with the current global trend towards a risk-based approach to reviewing aviation operations.

4.91      The committee notes with interest a submission from the Flight Safety Foundation, drawing attention to the Basic Aviation Risk Standard (BARS), an industry-based aviation standard originally developed to address higher risk aviation operations in the mining and resources sector.[75]

4.92      In brief, the BARS program tests operators' internal systems and processes against the risk standard, notes deficiencies and establishes correction action plans with defined close-out dates, after which their status is tracked. The second part of the process is an operational review:

...by the member organization of end-point high-risk activities. Rather than include these in a broad based audit once per year, these may be conducted independently of the BARS audit and at a suitable frequency. Combined with a BARS audit, this becomes a more effective means of identifying and reviewing key operational risks.[76]

4.93      The resulting controls, the committee was informed, can often be higher than those prescribed by national regulations.[77]

4.94      The ATSB currently assesses whether operators comply with rules when deciding the scope of accident investigations and what action to take, with no attempt to assess or report on whether the rules are appropriate for the nature of the operation.[78] The committee believes that the agency would do better if it had access to operator risk profiles, which would in turn produce a better investigative outcome for all passengers. This, the committee believes, is where agreed standards derived from industry and CASA could be useful by providing an accepted standard against which both operational audits and accident investigations could be conducted.

Quality control

4.95      Given that the ATSB investigation and reporting process is currently open to subjective analysis and review by the ATSB executive, the committee is of the view that an independent quality control system is desirable and should be established. Such a system would go a long way to increasing public and industry confidence. It would also provide an independent advocate to indicate to the government when budget pressures, combined with workloads, are putting pressure on the ATSB to take shortcuts which are in breach of best practice and Australia's international obligations.

4.96      The committee does not wish to be overly prescriptive about the design of the quality control system, merely to recommend that one be implemented consistent with certain parameters outlined below.

Expert panel

4.97      The quality control mechanism should ideally have a panel of subject matter experts to draw from, which, to reduce the potential for conflicts of interest, would comprise recently retired practitioners who are well regarded by key stakeholders in the sector for their experience in the aviation industry, aviation accident investigation or aviation safety management. Since panel members' level of expertise must obviously be appropriate, recruitment processes would be stringent and regular refreshment of expertise mandatory. A regular turnover of panel members would ensure that experience is recent enough to be relevant and well regarded by industry. Such a system would provide a sustainable pool of expert knowledge. Panel members should be required to sign confidentiality agreements.

4.98      Establishing such a panel would provide an alternative to involving a new organisation such as the office of the Commonwealth Ombudsman or the Administrative Appeals Tribunal (AAT), and would be a more cost effective way of providing quality control, as has been proven in other aviation regulatory systems.

4.99      Following an incident or accident that triggers ATSB involvement, one or more panel members would provide advice about the appropriate scope of the ensuing ATSB investigation. At the end of the investigative process, the same panel members would review the resulting report and provide comments to the ATSB, prior to its being publicly released. It would ultimately remain the ATSB's responsibility to determine the scope of its investigation and the content and recommendations contained in its report. However, should a substantial difference of opinion arise, the panel's advice would be made available to the minister and the Parliament on request. This panel would also provide ATSB personnel and those involved in the DIP process a channel through which serious concerns about scope and evidence could be reviewed.

Parliamentary scrutiny

4.100         The system sketched out above would serve to provide the Parliament with a level of confidence by requiring the expert panel to, on request, provide their advice to the ATSB to both the minister and the Parliament for review. The committee envisages that this would only occur if a serious difference of opinion arose but it also provides the minister or the Parliament with a means of review. 

Recommendation 8

4.101         The committee recommends that an expert aviation safety panel be established to ensure quality control of ATSB investigation and reporting processes along the lines set out by the committee.

4.102         The committee was not made aware of any significant budget shortfall.[79] However, this approach outlined above will expose any resourcing gaps or issues. To address any resourcing issues which may arise, a process should be developed by which the ATSB could claim supplementary funding when the task load of accident investigation exceeds planned figures by an agreed margin.

Recommendation 9

4.103         The committee recommends that the government develop a process by which the ATSB can request access to supplementary funding via the minister.

Navigation: Previous Page | Contents | Next Page