When experts conflict

Parliament house flag post

When experts conflict

Posted 25/10/2013 by Luke Buckmaster

Making decisions about complex public policy issues inevitably involves the assistance of experts. On occasion, however, experts in a given area disagree in their judgements.

In such cases, how can non-experts go about deciding which experts to believe?

Expert disagreement is one of the issues addressed in a recent Parliamentary Library research paper, Expertise and public policy: a conceptual guide.

As noted in the paper, expert disagreement poses a problem because by definition non-experts are not in a strong position to decide which of the experts' judgements is the most correct.

Using social expertise

The paper argues that the only way that non-experts are able to appraise expertise and expert claims is through the use of social expertise. This is expertise using social judgements that enable them to determine who to believe when they are not in a position to judge what to believe.

The paper discusses four different types of evidence that a non-expert might consider in order to establish that the word of one expert is more credible than that of their rival.

This can be thought of as a framework for using social expertise to evaluate expert claims. The question is, what sort of framework does it provide? How well do each of its elements stand up to scrutiny?

Which expert seems the more credible?

This refers to one expert being able to demonstrate ‘dialectical superiority’ over the other in, for example, a debate. The emphasis is on how statements are presented, rather than the content of the statements themselves. Examples include the ability to provide rebuttals, quickness or smoothness of response and more clearly explaining the evidence presented.

One advantage of this approach is that it allows non-experts to apply the kinds of social judgment they use in everyday life in assessing credibility claims. However, such evidence may simply be the result of better debating skills or ‘stylistic polish’. It is also doubtful that any marks of dialectical superiority are universal. Rather, it seems more likely that they are dependent on subject matter and the context in which the arguments are presented.

Who has the numbers on their side?

A common strategy for justifying acceptance of one expert’s conclusions over a rival’s is that more experts agree with the former than the latter, for example, the argument that there is an overwhelming consensus among climate scientists that human activity is having an impact on the climate.

Arguably, non-experts are relatively well placed to employ the strategy of going by the numbers—understood as being able to ‘read the scientific consensus’ on a particular controversy.

A practical problem with this strategy is that a scientific consensus may not be available (the science may not yet be 'in'). Also, some question this approach on the grounds that experts on one side of an argument may not be sufficiently independent from one another. While others have suggested that this problem has been overstated, it is fair to say that 'going by the numbers' is an approach that is most justifiably used in cases where the numbers are substantially in favour of one side.

Are there any relevant interests or biases?

Another strategy in deciding between conflicting expert claims is evidence of distorting interest and biases (for example, pecuniary interests).Interests are ‘often one of the more accessible pieces of information that a novice can glean about an expert’.

There are two main objections to the conflict of interest approach.

First, we can't simply assume that interest will distort conclusions. Second, interest objections add little to attempts to evaluate the validity of an expert argument. The ability to demonstrate the distorting impact of a conflict of interest will be on the basis of the quality of the arguments raised, meaning one does not need the interest-objection.

At best, interest based arguments should probably be used as a prompt for closer examination of an expert’s claim.

What are the experts track records?

For non-experts, information about whether an expert has gotten things right in the past can provide insight into an expert's credibility.

However, instances in which a non-expert will have access to the track records of competing experts are not that common. Also, in some cases the ‘true’ outcome of an expert dispute can itself be highly contested. Further, a non-expert would be unwise to assume that an expert with a good track record could automatically be trusted on all matters into the future.

An ethic of social expertise?

By identifying the above strategies’ strengths and limitations, the paper suggests how each might best be used.

First, they can be used in combination to improve their strength and reliability. Second, non-experts are able to draw on advice from meta-experts(experts on experts), independent non-partisan expert bodies, and less formally, policy blogs focused on communicating technical policy areas to a non-expert audience.

The important point is that because non-experts possess social expertise they are potentially able to make reasoned judgements about expert claims on technical matters. This means that they are not necessarily passive recipients of expert activity.

However, if non-experts are to become more active in their use of experts and meta-experts (developing what might be called an ethic of social expertise), then they should strive to better understand expertise and be more systematic and considered in their use of social expertise.

*Co-authored with Matthew Thomas

Image source: Wikimedia Commons


  • 21/01/2014 1:44 PM
    Luke Buckmaster said:

    Hi Martin, thanks for your comment. The point you make relates to the discussion in the paper about whether the more fluent debater possesses greater expertise. The ability to translate technical language into ‘common sense’ may demonstrate a mastery of the subject but may also simply be the result of better communication skills or ‘stylistic polish’. Note also that the that the paper does stress the importance of combining methods for evaluating experts.

  • 21/01/2014 1:44 PM
    martin butterfield said:

    I believe I may have achieved "beer mat" expertise in the meta-expertise on expertise studies. The linked paper reminded me of an old definition of an expert "One who knows more and more about less and less." Rather than just a cheap, almost Luddite, shot that could be interpreted as implying that while they are verty knowledgeable about their field (eg world leaders on the biology of taxon 'x" ) they can't provide a coherent context to the rest of us why that is important and thus why (eg) the habitat for species x shouldn't be subject to some degrading development process. It seems this problem will be amplified if Interactional expertise, defined in the paper as being able to speak the language of the expert group, is seen as the peak achievement. I suggest that it is important that the valuable experts are also able to speak Common Sense and can cross-translate between these tongues. When comparing experts I would also add "Were they reliant on jargon?"

Thank you for your comment. If it does not require moderation, it will appear shortly.
Facebook LinkedIn Twitter Add | Email Print


Flagpost is a blog on current issues of interest to members of the Australian Parliament

Parliamentary Library Logo showing Information Analysis & Advice




refugees asylum immigration Australian foreign policy Parliament climate change elections women social security Indigenous Australians Australian Bureau of Statistics Employment Sport illicit drugs people trafficking taxation Medicare welfare reform Australian Defence Force higher education welfare policy United Nations Asia income management Middle East criminal law disability Australian Sports Anti-Doping Agency World Anti-Doping Agency United States federal budget health financing gambling school education forced labour aid statistics Australian Electoral Commission WADA emissions trading Australia in the Asian Century steroids detention Private health insurance OECD ASADA labour force transport Law Enforcement Australian Federal Police Industrial Relations people smuggling dental health National Disability Insurance Scheme Australian Crime Commission slavery Senate election results Papua New Guinea Australian Public Service International Women's Day corruption Afghanistan Fair Work Act child protection debt federal election 2013 parliamentary procedure poker machines ALP New Zealand Newstart Parenting Payment 43rd Parliament political parties Census constitution High Court skilled migration voting Federal Court terrorist groups Higher Education Loan Program HECS youth paid parental leave Aviation environment foreign debt gross debt net debt defence capability customs doping health crime health risks multiculturalism aged care Gonski Review of Funding for Schooling sex slavery sea farers Special Rapporteur leadership United Kingdom UK Parliament Electoral reform politics banking firearms public policy violence against women domestic violence mental health China ADRV terrorism social media pensions welfare ASIO intelligence community Australian Security Intelligence Organisation governance public service reform Carbon Pricing Mechanism carbon tax mining military history employer employee fishing by-election European Union same sex relationships international relations coal seam gas family assistance planning United Nations Security Council Australian economy food vocational education and training Drugs Indonesia children codes of conduct terrorist financing money laundering Productivity asylum seekers early childhood education Canada Population Financial sector national security fuel disability employment Tasmania integrity science research and development Australian Secret Intelligence Service sexual abuse federal state relations World Trade Organization Australia accountability housing affordability bulk billing water renewable energy children's health health policy Governor-General US economy export liquefied natural gas foreign bribery question time speaker superannuation expertise Senators and Members climate Intergovernmental Panel on Climate Change Department of Agriculture Fisheries and Forestry food labelling Pacific Islands reserved seats new psychoactive substances synthetic drugs UNODC carbon markets health reform Indigenous constitutional recognition of local government local government consumer laws PISA royal commission US politics language education baby bonus Leaders of the Opposition Parliamentary remuneration health system Australia Greens servitude Trafficking Protocol energy forced marriage rural and regional Northern Territory Emergency Response ministries social citizenship human rights citizenship Defence High Court; Indigenous; Indigenous Australians; Native Title ACT Indigenous education Norfolk Island External Territories emissions reduction fund; climate change child care funding refugees immigration asylum procurement Indigenous health e-voting internet voting nsw state elections 44th Parliament 2015 ABS Age Pension Death penalty capital punishment execution Bali nine Bali bombings Trade EU China soft power education Fiji India Disability Support Pension Antarctica Diplomacy by-elections state and territories workers Bills anti-corruption fraud bribery transparency corporate ownership whistleblower G20 economic reform innovation standards NATO Members of Parliament Scottish referendum Middle East; national security; terrorism social services Criminal Code Amendment (Misrepresentation of Age to a Minor) Bill 2013 online grooming sexual assault of minors ACT Assembly public health smoking plain packaging tobacco cigarettes Asia; Japan; international relations Work Health and Safety Migration; asylum seekers; regional processing China; United States; international relations fiscal policy Racial Discrimination Act; social policy; human rights; indigenous Australians Foreign policy Southeast Asia Israel Palestine regional unemployment asylum refugees immigration political finance donations foreign aid Economics efficiency human rights; Racial Discrimination Act employment law bullying Animal law; food copyright Australian Law Reform Commission industry peace keeping contracts workplace policies trade unions same-sex marriage disorderly conduct retirement Parliament House standing orders public housing prime ministers election timetable sitting days First speech defence budget submarines Somalia GDP forestry world heritage political engagement leave loading Trade; tariffs; safeguards; Anti-dumping public interest disclosure whistleblowing Productivity Commission regulation limitation period universities Ireland cancer gene patents genetic testing suspension of standing and sessional orders animal health live exports welfare systems infant mortality middle class welfare honorary citizen railways disciplinary tribunals standard of proof World Health Organisation arts international students skilled graduate visas temporary employment visas apologies roads Italy national heritage NHMRC nutrition anti-dumping Constitutional reform referendum Rent Assistance competition policy pharmaceutical benefits scheme obesity evidence law sacrament of confession US presidential election international days DFAT UN General Assembly deregulation Regulation Impact Statements administrative law small business Breaker Morant homelessness regional engagement social determinants of health abortion Youth Allowance Members suspension citizen engagement policymaking federal election 2010 workplace health and safety Trafficking in Persons Report marine reserves hearing TAFE Victoria astronomy resources sector YMCA youth parliament alcohol Korea rebate Australian Greens presidential nomination Racial Discrimination Act entitlements political parties preselection solar hot water Financial Action Taskforce Horn of Africa peacekeeping piracy Great Barrier Reef Stronger futures political financing Hung Parliament political education social inclusion Social Inclusion Board maritime early childhood National Quality Framework for Early Childhood Education and Care Murray-Darling Basin Iran sanctions Norway hospitals

Show all
Show less
Back to top