When experts conflict

Parliament house flag post

When experts conflict

Posted 25/10/2013 by Luke Buckmaster

Making decisions about complex public policy issues inevitably involves the assistance of experts. On occasion, however, experts in a given area disagree in their judgements.

In such cases, how can non-experts go about deciding which experts to believe?

Expert disagreement is one of the issues addressed in a recent Parliamentary Library research paper, Expertise and public policy: a conceptual guide.

As noted in the paper, expert disagreement poses a problem because by definition non-experts are not in a strong position to decide which of the experts' judgements is the most correct.

Using social expertise

The paper argues that the only way that non-experts are able to appraise expertise and expert claims is through the use of social expertise. This is expertise using social judgements that enable them to determine who to believe when they are not in a position to judge what to believe.

The paper discusses four different types of evidence that a non-expert might consider in order to establish that the word of one expert is more credible than that of their rival.

This can be thought of as a framework for using social expertise to evaluate expert claims. The question is, what sort of framework does it provide? How well do each of its elements stand up to scrutiny?

Which expert seems the more credible?

This refers to one expert being able to demonstrate ‘dialectical superiority’ over the other in, for example, a debate. The emphasis is on how statements are presented, rather than the content of the statements themselves. Examples include the ability to provide rebuttals, quickness or smoothness of response and more clearly explaining the evidence presented.

One advantage of this approach is that it allows non-experts to apply the kinds of social judgment they use in everyday life in assessing credibility claims. However, such evidence may simply be the result of better debating skills or ‘stylistic polish’. It is also doubtful that any marks of dialectical superiority are universal. Rather, it seems more likely that they are dependent on subject matter and the context in which the arguments are presented.

Who has the numbers on their side?

A common strategy for justifying acceptance of one expert’s conclusions over a rival’s is that more experts agree with the former than the latter, for example, the argument that there is an overwhelming consensus among climate scientists that human activity is having an impact on the climate.

Arguably, non-experts are relatively well placed to employ the strategy of going by the numbers—understood as being able to ‘read the scientific consensus’ on a particular controversy.

A practical problem with this strategy is that a scientific consensus may not be available (the science may not yet be 'in'). Also, some question this approach on the grounds that experts on one side of an argument may not be sufficiently independent from one another. While others have suggested that this problem has been overstated, it is fair to say that 'going by the numbers' is an approach that is most justifiably used in cases where the numbers are substantially in favour of one side.

Are there any relevant interests or biases?

Another strategy in deciding between conflicting expert claims is evidence of distorting interest and biases (for example, pecuniary interests).Interests are ‘often one of the more accessible pieces of information that a novice can glean about an expert’.

There are two main objections to the conflict of interest approach.

First, we can't simply assume that interest will distort conclusions. Second, interest objections add little to attempts to evaluate the validity of an expert argument. The ability to demonstrate the distorting impact of a conflict of interest will be on the basis of the quality of the arguments raised, meaning one does not need the interest-objection.

At best, interest based arguments should probably be used as a prompt for closer examination of an expert’s claim.

What are the experts track records?

For non-experts, information about whether an expert has gotten things right in the past can provide insight into an expert's credibility.

However, instances in which a non-expert will have access to the track records of competing experts are not that common. Also, in some cases the ‘true’ outcome of an expert dispute can itself be highly contested. Further, a non-expert would be unwise to assume that an expert with a good track record could automatically be trusted on all matters into the future.

An ethic of social expertise?

By identifying the above strategies’ strengths and limitations, the paper suggests how each might best be used.

First, they can be used in combination to improve their strength and reliability. Second, non-experts are able to draw on advice from meta-experts(experts on experts), independent non-partisan expert bodies, and less formally, policy blogs focused on communicating technical policy areas to a non-expert audience.

The important point is that because non-experts possess social expertise they are potentially able to make reasoned judgements about expert claims on technical matters. This means that they are not necessarily passive recipients of expert activity.

However, if non-experts are to become more active in their use of experts and meta-experts (developing what might be called an ethic of social expertise), then they should strive to better understand expertise and be more systematic and considered in their use of social expertise.

*Co-authored with Matthew Thomas

Image source: Wikimedia Commons


  • 21/01/2014 1:44 PM
    Luke Buckmaster said:

    Hi Martin, thanks for your comment. The point you make relates to the discussion in the paper about whether the more fluent debater possesses greater expertise. The ability to translate technical language into ‘common sense’ may demonstrate a mastery of the subject but may also simply be the result of better communication skills or ‘stylistic polish’. Note also that the that the paper does stress the importance of combining methods for evaluating experts.

  • 21/01/2014 1:44 PM
    martin butterfield said:

    I believe I may have achieved "beer mat" expertise in the meta-expertise on expertise studies. The linked paper reminded me of an old definition of an expert "One who knows more and more about less and less." Rather than just a cheap, almost Luddite, shot that could be interpreted as implying that while they are verty knowledgeable about their field (eg world leaders on the biology of taxon 'x" ) they can't provide a coherent context to the rest of us why that is important and thus why (eg) the habitat for species x shouldn't be subject to some degrading development process. It seems this problem will be amplified if Interactional expertise, defined in the paper as being able to speak the language of the expert group, is seen as the peak achievement. I suggest that it is important that the valuable experts are also able to speak Common Sense and can cross-translate between these tongues. When comparing experts I would also add "Were they reliant on jargon?"

Thank you for your comment. If it does not require moderation, it will appear shortly.
Facebook LinkedIn Twitter Add | Email Print


Flagpost is a blog on current issues of interest to members of the Australian Parliament

Parliamentary Library Logo showing Information Analysis & Advice




refugees asylum immigration Australian foreign policy Parliament elections climate change social security women welfare reform taxation Indigenous Australians Australian Defence Force welfare policy school education higher education private health insurance health financing emissions trading Senate Australian Bureau of Statistics employment people trafficking Asia statistics Middle East illicit drugs gambling health reform federal election 2010 Australian Public Service income management Medicare disability Sport United Nations environment industrial relations constitution transport politics criminal law Afghanistan health forced labour food public service reform aged care aid Australian Sports Anti-Doping Agency World Anti-Doping Agency United States federal budget Carbon Pricing Mechanism dental health international relations governance regulation Fair Work Act voting law enforcement electoral reform OECD Australian Electoral Commission WADA child protection poker machines Australia in the Asian Century steroids National Disability Insurance Scheme detention 43rd Parliament slavery health system leadership domestic violence parliamentary procedure International Women's Day accountability defence capability multiculturalism ASADA Australian Federal Police labour force people smuggling debt New Zealand Australian Crime Commission pharmaceutical benefits scheme political parties coal seam gas Human rights crime China Census election results UK Parliament Papua New Guinea banking corruption pensions children's health Aviation federal election 2013 foreign debt gross debt net debt Senators and Members ALP Newstart Parenting Payment Youth Allowance sea farers United Kingdom energy food labelling Australian economy violence against women vocational education and training military history by-election High Court skilled migration mental health Federal Court terrorist groups science social media Higher Education Loan Program HECS federal state relations youth paid parental leave same sex relationships customs planning doping health risks Gonski Review of Funding for Schooling sex slavery Special Rapporteur Northern Territory Emergency Response social policy ANZUS Rural and regional trade unions Foreign affairs election timetable Indigenous royal commission Productivity firearms public policy Population ADRV terrorism transparency research and development welfare ASIO intelligence community Australian Security Intelligence Organisation carbon tax mining employer employee renewable energy regional unemployment fishing European Union family assistance United Nations Security Council forestry Drugs welfare systems Indonesia children Constitutional reform local government codes of conduct terrorist financing homelessness Parliamentary remuneration money laundering Trafficking in Persons Report social inclusion paternalism environmental law US presidential election nutrition ODA Defence sitting days electoral divisions Southeast Asia administrative law universities TAFE Ireland citizenship asylum seekers early childhood education Canada Financial sector national security fuel disability employment Tasmania integrity standards NATO Australian Secret Intelligence Service sexual abuse World Trade Organization Australia public health housing affordability bulk billing water health policy Governor-General US economy export liquefied natural gas foreign bribery question time speaker superannuation public housing expertise climate Intergovernmental Panel on Climate Change Department of Agriculture Fisheries and Forestry Pacific Islands reserved seats new psychoactive substances synthetic drugs UNODC carbon markets animal health middle class welfare constitutional recognition of local government referendum consumer laws PISA competition policy US politics language education baby bonus Leaders of the Opposition citizen engagement policymaking Australia Greens servitude Trafficking Protocol forced marriage alcohol entitlements ministries Hung Parliament social citizenship maritime Iran regional students school chaplains federal budget 2011-12 salary Medicare Locals primary care Building the Education Revolution marine pollution sustainability prisons police deaths in custody electoral margins electoral pendulum electoral redistribution redistribution NSW redistribution WA redistribution ACT electoral boundaries ASEAN Sustainable Development Goals Double dissolution Senators safety vehicles MYEFO Pathology tertiary education Taiwan Xi Ma meeting family violence government financial advisers financial planners Financial System Inquiry Murray Inquiry China; Economic policy; Southeast Asia; Africa housing Speaker; House of Representatives; Parliament High Court; Indigenous; Indigenous Australians; Native Title ACT Indigenous education Norfolk Island External Territories emissions reduction fund; climate change child care funding refugees immigration asylum procurement Indigenous health e-voting internet voting nsw state elections 44th Parliament 2015 ABS Age Pension Death penalty capital punishment execution Bali nine Bali bombings Trade EU China soft power education Fiji India Disability Support Pension Antarctica Diplomacy by-elections state and territories Bills anti-corruption fraud bribery corporate ownership whistleblower G20 economic reform innovation Members of Parliament Scottish referendum Middle East; national security; terrorism social services Criminal Code Amendment (Misrepresentation of Age to a Minor) Bill 2013 online grooming sexual assault of minors ACT Assembly smoking plain packaging tobacco cigarettes Asia; Japan; international relations Work Health and Safety Migration; asylum seekers; regional processing China; United States; international relations fiscal policy Racial Discrimination Act; social policy; human rights; indigenous Australians Foreign policy Israel Palestine asylum refugees immigration political finance donations foreign aid Economics efficiency human rights; Racial Discrimination Act employment law bullying Animal law; food copyright Australian Law Reform Commission industry peace keeping contracts workplace policies same-sex marriage disorderly conduct retirement Parliament House standing orders prime ministers First speech defence budget submarines workers Somalia GDP world heritage political engagement leave loading Trade; tariffs; safeguards; Anti-dumping public interest disclosure whistleblowing Productivity Commission limitation period cancer gene patents genetic testing suspension of standing and sessional orders live exports infant mortality honorary citizen railways disciplinary tribunals standard of proof World Health Organisation arts international students skilled graduate visas temporary employment visas apologies roads Italy national heritage NHMRC anti-dumping Rent Assistance obesity evidence law sacrament of confession international days DFAT UN General Assembly deregulation Regulation Impact Statements small business Breaker Morant regional engagement social determinants of health abortion Members suspension workplace health and safety marine reserves hearing Victoria astronomy resources sector YMCA youth parliament Korea rebate Australian Greens presidential nomination Racial Discrimination Act political parties preselection solar hot water Financial Action Taskforce Horn of Africa peacekeeping piracy Great Barrier Reef Stronger futures political financing political education Social Inclusion Board early childhood National Quality Framework for Early Childhood Education and Care Murray-Darling Basin sanctions Norway hospitals republic President Barack Obama Presidential visits qantas counselling Korean peninsula Work Choices biosecurity hendra federalism federation preselection therapeutic goods Therapeutic Goods Administration plebiscites computer games pests suicide nuclear COAG Ministerial Councils floods ADHD stimulant medication advertising electricity extradition conscience votes poverty preventative health rural health coastal erosion Parliamentary Budget Office work-life balance

Show all
Show less
Back to top