4.1
Underpinning critiques of the scientific base for land-based run-off and Reef science more broadly is a concern about the peer review process and the replicability of scientific research. This critique forms the basis of calls for an independent audit of Reef science. These concerns were shared by a number of other submitters to this inquiry, including agricultural peak bodies, farming groups and individuals.
4.2
This chapter considers the views of witnesses and submitters that question the reliability of the peer review process, and views that there is a replicability crisis within the Reef science field. This chapter then considers the scientific community's response to those criticisms.
Peer review process and the replication crisis
4.3
The peer review process ensures the quality and reliability of scientific research and according to the Australian Academy of Science (the Academy) 'is a central component of the scientific process'. The first step of this process involves the selection of scientific peers 'with comparable skills and knowledge' to assess the scientific merit of a research paper prior to its publication.
4.4
After publication, the paper then 'becomes part of the public record and is subject to intense scrutiny by thousands of scientists – the discipline as a whole'. The Academy submitted that this discipline as a whole process enables flaws in research to be identified and addressed in subsequent studies, with subsequent observations and conclusions informing:
…new hypotheses, which are then tested and published in the same way. This continual analysis leads ultimately to robust conclusions. There are few other forms of knowledge that are subject to this intensity of scrutiny.
4.5
However, this peer review process has been criticised by a number of scientists and non-government bodies, including Dr Ridd. Dr Ridd submitted that the peer review process for many disciplines of science has serious flaws, and is an issue accepted by most major national academies of science. One potential flaw identified is 'that it almost guarantees groupthink, and can often exclude views from dissenting scientists'. Further, the committee heard that the peer review process fails to identify whether evidence within research papers is incorrect and that these major errors are found in the work produced by Reef science institutions. According to Dr Ridd, there is a reluctance within those institutions to address the problem:
…there is a general reluctance of the institutions to rectify problems. They are in denial about their serious deficiency of Quality Assurance protocols. In some cases, they actively cover up problems, and vilify or exclude those who raise concerns.
4.6
For this reason, Dr Ridd is of the view that there is considerable doubt that Australia's Reef science institutions provide reliable scientific evidence; however, whilst this doubt 'does not imply all their work is wrong…we cannot conclude that most individual parts of the scientific evidence, or the "consensus" documents, are reliable'.
4.7
According to critics of Reef science, this doubt in science is driven by a replication crisis. Dr Ridd's evidence referenced research that claims 'a very large part of the scientific literature, perhaps half, is flawed' and made reference to the Australian Chief Scientist expressing his concern about this issue. Dr Ridd critiqued those scientific institutions that believe the peer review process 'is a marvellous quality assurance system', instead stating that the evidence from other areas in science has shown that it is not. Acknowledging that he is in the 'minority on whether or not the reef is being damaged by farms', he added that he is in the majority when it comes to his concerns about the quality assurance problems in science.
4.8
Dr Ridd accused Reef scientists of dismissing the replication crisis by arguing that it is a problem of other fields of research but not Reef science. Dr Ridd suggested that it was either arrogance or blind optimism and:
…considering that the potentially erroneous science of the [Reef] is affecting every major industry in North Eastern Australia, it is very surprising that such a strenuous argument was made to NOT do a little more checking. How can one argue against a little bit of checking?
4.9
Various agriculture groups and the Institute of Public Affairs shared the view of there being a replication crisis in Reef science. A number made reference to issues of corruption in research institutes with the release of the Queensland Crime and Corruption Commission report, Reducing the risk of research fraud.
4.10
In order to respond to the replication crisis, Dr Ridd (along with other likeminded academics) and various representatives from the agricultural sector have called for funding ($5 million) to establish a body consisting of independent scientists not associated with a government institution to audit the scientific research associated with the Reef. Supporters argue that a full audit of the evidence relating to agriculture and the Reef would take approximately two years and better position governments to 'make decisions on a more solid scientific base'.
4.11
Dr Ridd, who is the primary advocate for establishing a separate audit body called the Office of Science Quality Assurance (the Office), advised the committee that having an existing Reef research body reviewing the science 'would be Caesar judging Caesar and would be an absolute disaster'. Instead, Dr Ridd called for the Office being run through the Australian National Audit Office. The Office would take information, such as coral growth rates, and subject the data to real scrutiny and determine whether the experiments get the same result. As a result, 'scientists would be much more careful about making statements and publishing results, where there was a likelihood that they haven't been as careful as they should have been'. A similar view was shared by the Institute of Public Affairs, that recommended:
…that there need to be mechanisms established whereby assessments of evidence for studies of the impact of farm water runoff on the health of the Great Barrier Reef and catchment areas are produced by suitably qualified scientists who can understand the relevant scientific studies but have a detachment in making assessments of causation. The results of these reviews that can then be published in the peer-reviewed literature and also presented within a public forum and open to debate.
4.12
The Australian Banana Growers' Council, whilst calling for 'water quality science being demystified for growers', was 'not convinced that a new office science quality assurance would be the silver bullet to improving the robustness of water quality science', rather it would 'add a new level of bureaucracy and more confusion'.
Science community's response
4.13
The scientific community, as well as government and non-government bodies, collectively defended the Reef science and the scientific process. These submitters and witnesses rejected the notion that there is a replication crisis within the ecological field, that the peer review process is inadequate in that it excludes dissenting views, and therefore a new quality science assurance oversight body is needed.
4.14
Concerning arguments of there being a replication crisis within Reef science, the committee heard that this concern originated in the field of psychology and medical science, which differs from the scientific practices applied to Reef science. Griffith University argued that ecological sciences differed from other areas of research due it being a slower evolving type of field than other fields such as medical research, which reduced its vulnerability to replication issues. Regarding specific concerns about the replicability of an Australia Institute of Marine Science (AIMS) study into coral growth rates, AIMS stated that it would not be useful to conduct the study again because it had already supplemented that work, and that it would be 'very, very expensive work'.
4.15
Scientific representatives denied this accusation that the peer review process excludes voices that challenge consensus views. Instead, they argued the scientific process promotes it. For those who hold this view, the Academy strongly encouraged them 'to submit and test their knowledge through the peer review process' because:
…if, in fact, the evidence they have, or they're purporting to have, is contrary to what we understand a particular situation to be it must be put through that peer review process so that knowledge can evolve. We want that level of debate, and the appropriate place, forum and methodology for that is the peer review process. I would encourage you to ask those people to test their knowledge in that forum.
4.16
TropWATER 'completely disagreed' with the idea for a need to subject Reef science to a more rigorous checking, testing and replication system. TropWATER argued that the system in place is sufficient and is designed to ensure scientist are always checking the work of their colleagues.
4.17
Regarding the specific concerns made by Dr Ridd, AIMS advised the committee that it had been in communication with Dr Ridd on numerous occasions; however:
…he doesn't seem to want to listen to any of the new information we give him. He's got a monotonic view on this. No matter what information or additional papers and studies we provide him with, he seems to ignore it all. I don't really have anything else to say about that.
4.18
Overall, the committee heard numerous expressions of support for the soundness of the existing scientific process, namely the peer review process. Dr Peter Doherty, a member of the Independent Science Panel for the Water Quality Improvement Plan (WQIP) reflected that 40 years working as a scientist and research manager has not 'in fact left me disillusioned with the peer view process'. Dr Doherty took the view that the current process provides 'the best available synthesis of information and evidence from any question to which we choose to point it', referencing the Consensus Statement as the best available summary of information about the Reef and the water quality issue. His colleague, Dr Roger Shaw disagreed with the assumption that the peer review process is poor, instead emphasised the difficulty of replicating science 'in a changing ecosystem' due to 'massive episodic flood events and climate, including cyclones'. He added that Nature journal, which receives 27,000 scientific papers each week, only withdraws 0.27 per cent of those papers. For this reason, Dr Shaw is of the view that 'the general process is handling it fairly well, but things will slip through'. Dr Garrett highlighted the importance of a scientist's reputation, and for this reason, under the peer review process scientists 'put a lot of time and effort into making sure that…they are accurate'.
4.19
Regarding the importance of the peer review process to inform public policy, the Department of Agriculture, Water and the Environment made clear of its 'absolute and fundamental reliance' on peer reviewed science, which is the 'foundation for the policy advice' it provides to government.
4.20
Whether the science is corrupted, defective or concocted, the various scientific bodies recognised there is always a risk, which the scientific community took seriously; however, the peer review process itself provides the necessary check and balances to ensure the accuracy of results. To further ensure the integrity, research institutions have a range of codes of conducts, guides of behaviour and ethics processes. Overall, the committee heard that this type of conduct is generally infinitesimal and 'the exception and not the rule'.
4.21
Various scientific representatives discussed the occurrence of cherry-picking research and data by those who reject the existing scientific base, especially within the context of controversial public policy decisions. The Academy submitted that good science leads to better public policy; however, when people are opposed to a policy there is a tendency to 'cherry pick' scientific findings, which according to the Academy is 'antithetical to good policy'. The Academy explained that 'rather than consulting and analysing the body of literature systematically', opponents to the scientific process will cherry-pick 'evidence to support a decision or position is dangerous and leads to poor judgement and outcomes'.
4.22
This cherry-picking of evidence, according to Professor Ian Chubb, is to deny the evidence, which in his view is absurd and ludicrous 'because we can respect the science, we can accept the science, we can question the science and we can question legitimately where somebody…thinks there is a gap in the science'; however, 'to cherry-pick the evidence that is available and then use that to deny the science seems to me to be a seriously backward step'.
4.23
In a letter to Australian and Queensland government ministers, Professor Chubb on behalf of the Independent Expert Panel iterated its 'members' unanimous support for the science that underpins the evidence of the extent and probable causes of damage to the [Reef]'. Referencing Dr Ridd's 'roadshow' and the wider agricultural industry's response to the Queensland Government's Reef regulations package, Professor Chubb wrote:
We do not have a particular view on the regulations, which is outside our remit, but we chose not to sit by and watch the science being disputed and sometimes misrepresented.
…
It is our advice to you that the science we have seen and discussed during our fifteen meetings has been conducted according to the most rigorous and widely accepted processes employed by professional scientists.
4.24
The Independent Expert Panel, and Professor Chubb during his appearance at the public hearing, warned that those opposing the Reef science are using the tactic of sowing doubt to undermine the scientific basis for Reef protection, which is a tactic used by the tobacco industry to undermine health policy. A subsequent tactic is 'to invoke the notion of conspiracy of the world's scientists all working together to stop the outsider getting their results published, or accepted'. Professor Chubb, who emphasised that whilst exploring a gap in scientific evidence is a 'perfectly appropriate part of the scientific process', added that tactics used by those against the existing scientific process is inappropriate. The Academy also noted previous occurrences of discrediting the underlying science to achieve a policy outcome and its impact on trust:
We have seen many occurrences over time where there has been an attempt to discredit the underlying science, not because the science has necessarily been worthy of discrediting but perhaps because a policy outcome or decision hasn't been one that's been met with consensus and so the easy target has been to discredit the underlying science. It's unfortunate because the seeding of that doubt does erode trust.
4.25
Universities Australia made clear of the vital role debate has both within the scientific process and concerning 'policy design and its implementation arising from science'; however, argued it was not 'legitimate to denigrate the robust and international research methodology that underpins that science'.
4.26
To further highlight the shared concern of the tactics used to manufacture doubt in the scientific process, Professor Chubb referenced an article that detailed Dr Ridd's appearance at a conference in the Netherlands. At this conference, Dr Ridd reportedly outlined the tactics used to undermine both Reef and climate science. Dr Ridd is quoted to have said:
"We're using the replication crisis to convince the politicians that, you know, if you've got a problem in medical science, why haven't you got a problem with Great Barrier Reef science,"…"The replication crisis is a dagger at the heart of the climate alarmists and Great Barrier Reef alarmists"
…
"I think it is … easier to show the reef is not in trouble, than to demonstrate unequivocally that the climate change thing is a total myth," … "If we can suddenly change everybody's mind that actually the reef is fine, it will get them thinking ... 'When they've been telling us [the reef is dying] for so many years, then what about all these other things we've been told?'"
4.27
In response to criticisms of the peer review process, Professor Chubb wrote that those people:
…usually imply that the two or three ‘experts’ reviewing manuscripts presented for publication can be wrong, or biased in favour of a particular outcome. Of course, that is possible. An essential part of the peer review process, however, is the intense scrutiny of publications and reports by numerous experts before and after publication. That scrutiny can extend beyond publications, to data sets, computer code and software.
4.28
Professor Chubb outlined the scientific and peer review process, which includes publications and reports being reviewed and evaluated by multiple ('possibly hundreds and sometimes thousands') of experts and how pre-existing ideas and research are used to form hypotheses for experimental design. Through this process, '[f]laws can be revealed, and it is part of the process of science to expose any such flaws and to put that, too, into the public domain for a similar level of scrutiny'. A result of this ongoing scientific practice is that knowledge from any field of science is a product of 'accumulated evidence that survives intense scrutiny and has not been disproven'. Scientific findings and conclusions are a subject to further scrutiny prior to their adoption as 'evidence-based policy, programs or regulations'. For this reason, the Independent Expert Panel's advice to the Australian and Queensland governments is 'that the science as we currently know it is robust and the conclusions appropriate'. In conclusion, the Independent Expert Panel made clear that:
…given the direction that quality science points, suitable policy and focussed regulation are both essential if we are to give the [Reef] a decent chance of survival in anything like its historically recognisable form.
We will learn more, and we will get more knowledge. But we have more than enough to know that without action to address climate change and to improve water quality the risk to the [Reef] is substantial. It is our view that the matter requires urgent attention and that policy development should not be distracted from the goal.
The evidence is strong, the science robust, the conclusions drawn from the science are sound.
Serious effort is needed to address the state of the GBR, and the effort is needed now to ensure that future generations will have a GBR to enjoy.
4.29
In a statement provided by Chief Scientist, Dr Alan Finkel, the committee heard of his 'confidence in the scientific process' and that 'there is every reason to believe that the scientific evidence that has informed [the Queensland Government's] policies and regulations is sound'. Dr Finkel's statement's proceeds to explain the reason for his confidence is because of the scientific process:
My confidence is based on the nature of the scientific process. Science is a global endeavour, with some eight million researchers working on ways to improve our way of life and how we interact with the world and each other. Science is already delivering in spades. Indeed, it's difficult to keep up with the steady stream of tangible advances that improve our health, increase our farm yields and contribute to our prosperity. The work I've been doing is about making science even better. In the scientific process, studies are placed under intense scrutiny and interrogation by the scientific community. Publication and the peer-review process are based on subject matter experts rigorously reviewing the scientific method against a background of deep knowledge. Time and again the scientific process has proven itself to be fundamentally sound, effective and self improving, but there is always scope in any field, business or organisation to improve the quality of the outputs.
4.30
Concerning references made by Dr Ridd about Dr Finkel's views about the peer review process, Dr Finkel clarified that he has 'spoken publicly to researchers on the importance of improving our already high quality and trustworthy research' and has proposed 'a rigorous quality assurance system for assessing the publication process'. However, this proposal 'should not be taken as a statement on the quality of published research but rather ensuring that publishers adhere to agreed publishing standards'. Subsequently, Dr Finkel does not support the proposal for a new office of science quality assurance. Rather, 'the proven scientific process, with its significant levels of internal and external scrutiny, should be relied upon as the most credible source of evidence to serve as an input into the policymaking process'.
4.31
Dr Garrett and Professor Chubb also rejected the idea of creating a new oversight body to review science. They argued that mechanisms were already in place to review science through the existing peer review process, and various oversight bodies with a wide range of backgrounds and expertise. Professor Chubb concluded the notion was 'ill-advised, ill-though-through, political and of limited value in any sensible discussion'.
Committee view
4.32
Of all the themes explored by the committee during the inquiry, the committee is extremely concerned by the unsubstantiated claims made by various witnesses and submitters about the adequacy of the peer review process. Those who hold this view questioned the reliability and replicability of the science produced by the well-established, peer review process.
4.33
In line with the Australian government, the committee does not share the view that the peer review process is inadequate or lacking, that it produces group think or corrupt behaviour, nor that it has created a replicability crisis within Reef science. Further, the committee does not believe that Australia's scientific community is intentionally covering up flaws in research and vilifying or excluding those scientists that raise concerns with the quality of the research. Rather, the committee is reassured by those scientific representatives that spoke of the sector's incorporation of dissenting and alternative views, and their encouragement for those scientists to submit their work through the peer review process. For this reason, the committee is not supportive of the proposal for the Office of Science Quality Assurance. The committee agrees that whilst more can be done to demystify the science that underpins governments' policy decisions, an additional oversight body would not improve the robustness of water quality science, and would be an expensive and politicised endeavour.