When experts conflict

Parliament house flag post

When experts conflict

Posted 25/10/2013 by Luke Buckmaster



Making decisions about complex public policy issues inevitably involves the assistance of experts. On occasion, however, experts in a given area disagree in their judgements.

In such cases, how can non-experts go about deciding which experts to believe?

Expert disagreement is one of the issues addressed in a recent Parliamentary Library research paper, Expertise and public policy: a conceptual guide.

As noted in the paper, expert disagreement poses a problem because by definition non-experts are not in a strong position to decide which of the experts' judgements is the most correct.

Using social expertise

The paper argues that the only way that non-experts are able to appraise expertise and expert claims is through the use of social expertise. This is expertise using social judgements that enable them to determine who to believe when they are not in a position to judge what to believe.

The paper discusses four different types of evidence that a non-expert might consider in order to establish that the word of one expert is more credible than that of their rival.

This can be thought of as a framework for using social expertise to evaluate expert claims. The question is, what sort of framework does it provide? How well do each of its elements stand up to scrutiny?

Which expert seems the more credible?

This refers to one expert being able to demonstrate ‘dialectical superiority’ over the other in, for example, a debate. The emphasis is on how statements are presented, rather than the content of the statements themselves. Examples include the ability to provide rebuttals, quickness or smoothness of response and more clearly explaining the evidence presented.

One advantage of this approach is that it allows non-experts to apply the kinds of social judgment they use in everyday life in assessing credibility claims. However, such evidence may simply be the result of better debating skills or ‘stylistic polish’. It is also doubtful that any marks of dialectical superiority are universal. Rather, it seems more likely that they are dependent on subject matter and the context in which the arguments are presented.

Who has the numbers on their side?

A common strategy for justifying acceptance of one expert’s conclusions over a rival’s is that more experts agree with the former than the latter, for example, the argument that there is an overwhelming consensus among climate scientists that human activity is having an impact on the climate.

Arguably, non-experts are relatively well placed to employ the strategy of going by the numbers—understood as being able to ‘read the scientific consensus’ on a particular controversy.

A practical problem with this strategy is that a scientific consensus may not be available (the science may not yet be 'in'). Also, some question this approach on the grounds that experts on one side of an argument may not be sufficiently independent from one another. While others have suggested that this problem has been overstated, it is fair to say that 'going by the numbers' is an approach that is most justifiably used in cases where the numbers are substantially in favour of one side.

Are there any relevant interests or biases?

Another strategy in deciding between conflicting expert claims is evidence of distorting interest and biases (for example, pecuniary interests).Interests are ‘often one of the more accessible pieces of information that a novice can glean about an expert’.

There are two main objections to the conflict of interest approach.

First, we can't simply assume that interest will distort conclusions. Second, interest objections add little to attempts to evaluate the validity of an expert argument. The ability to demonstrate the distorting impact of a conflict of interest will be on the basis of the quality of the arguments raised, meaning one does not need the interest-objection.

At best, interest based arguments should probably be used as a prompt for closer examination of an expert’s claim.

What are the experts track records?

For non-experts, information about whether an expert has gotten things right in the past can provide insight into an expert's credibility.

However, instances in which a non-expert will have access to the track records of competing experts are not that common. Also, in some cases the ‘true’ outcome of an expert dispute can itself be highly contested. Further, a non-expert would be unwise to assume that an expert with a good track record could automatically be trusted on all matters into the future.

An ethic of social expertise?

By identifying the above strategies’ strengths and limitations, the paper suggests how each might best be used.

First, they can be used in combination to improve their strength and reliability. Second, non-experts are able to draw on advice from meta-experts(experts on experts), independent non-partisan expert bodies, and less formally, policy blogs focused on communicating technical policy areas to a non-expert audience.

The important point is that because non-experts possess social expertise they are potentially able to make reasoned judgements about expert claims on technical matters. This means that they are not necessarily passive recipients of expert activity.

However, if non-experts are to become more active in their use of experts and meta-experts (developing what might be called an ethic of social expertise), then they should strive to better understand expertise and be more systematic and considered in their use of social expertise.

*Co-authored with Matthew Thomas

Image source: Wikimedia Commons

Comments

  • 21/01/2014 1:44 PM
    Luke Buckmaster said:

    Hi Martin, thanks for your comment. The point you make relates to the discussion in the paper about whether the more fluent debater possesses greater expertise. The ability to translate technical language into ‘common sense’ may demonstrate a mastery of the subject but may also simply be the result of better communication skills or ‘stylistic polish’. Note also that the that the paper does stress the importance of combining methods for evaluating experts.

  • 21/01/2014 1:44 PM
    martin butterfield said:

    I believe I may have achieved "beer mat" expertise in the meta-expertise on expertise studies. The linked paper reminded me of an old definition of an expert "One who knows more and more about less and less." Rather than just a cheap, almost Luddite, shot that could be interpreted as implying that while they are verty knowledgeable about their field (eg world leaders on the biology of taxon 'x" ) they can't provide a coherent context to the rest of us why that is important and thus why (eg) the habitat for species x shouldn't be subject to some degrading development process. It seems this problem will be amplified if Interactional expertise, defined in the paper as being able to speak the language of the expert group, is seen as the peak achievement. I suggest that it is important that the valuable experts are also able to speak Common Sense and can cross-translate between these tongues. When comparing experts I would also add "Were they reliant on jargon?"


Thank you for your comment. If it does not require moderation, it will appear shortly.

Add your comment

[Click to expand]

We welcome your comments, or additional information which is relevant to a post. These can be added by clicking on the ‘Add your comment’ option above. Please note that the Parliamentary Library will moderate comments, and reserves the right not to publish comments that are inconsistent with the objectives of FlagPost. This includes spam, profanity and personal abuse, as well as comments that are factually incorrect or politically partisan. We will close comments after three months.




Captcha
Generate a new image
Type characters from the image:

Facebook LinkedIn Twitter Add | Email Print

FlagPost

Flagpost is a blog on current issues of interest to members of the Australian Parliament


Parliamentary Library Logo showing Information Analysis & Advice

Archive

Syndication

Tagcloud

Refugees asylum climate change immigration Australian foreign policy parliament social security welfare policy elections welfare reform school education Australian Defence Force health financing higher education emissions trading indigenous Australians women private health insurance people trafficking Employment illicit drugs gambling health reform federal election 2010 Australian Bureau of Statistics statistics United Nations Asia Afghanistan disability income management Middle East Medicare sport health forced labour United States federal budget Industrial Relations Carbon Pricing Mechanism politics dental health criminal law transport aid child protection environment poker machines Australia in the Asian Century Australian Sports Anti-Doping Agency steroids World Anti-Doping Agency National Disability Insurance Scheme detention aged care 43rd Parliament slavery health system Law Enforcement Australian Federal Police Fair Work Act Australian Public Service governance labour force people smuggling debt taxation international relations constitution New Zealand food WADA Australian Crime Commission pharmaceutical benefits scheme corruption pensions public service reform children's health Aviation foreign debt gross debt net debt defence capability parliamentary procedure Senate Senators and Members ALP ASADA Newstart Parenting Payment multiculturalism Youth Allowance sea farers terrorist groups science social media Higher Education Loan Program HECS federal state relations accountability Papua New Guinea youth paid parental leave same sex relationships coal seam gas customs planning federal election 2013 Australian Electoral Commission doping OECD crime health risks International Women's Day Gonski Review of Funding for Schooling sex slavery Special Rapporteur Northern Territory Emergency Response social policy terrorism transparency research and development Mental health welfare ASIO intelligence community Australian Security Intelligence Organisation carbon tax mining High Court military history electoral reform employer employee renewable energy regional unemployment fishing European Union Federal Court family assistance skilled migration banking United Nations Security Council Australian economy forestry food labelling vocational education and training Drugs UK Parliament welfare systems Indonesia children Constitutional reform local government codes of conduct terrorist financing homelessness Parliamentary remuneration money laundering Trafficking in Persons Report energy social inclusion human rights paternalism integrity standards NATO Australian Secret Intelligence Service sexual abuse World Trade Organization Australia public health China housing affordability bulk billing political parties water productivity health policy Governor-General US economy trade unions domestic violence export liquefied natural gas foreign bribery firearms question time speaker superannuation public housing election results by-election expertise public policy climate Intergovernmental Panel on Climate Change leadership voting Department of Agriculture Fisheries and Forestry regulation Pacific Islands reserved seats new psychoactive substances synthetic drugs UNODC carbon markets animal health middle class welfare ADRV Census Indigenous constitutional recognition of local government referendum consumer laws PISA competition policy royal commission US politics violence against women language education baby bonus Leaders of the Opposition citizen engagement policymaking Australia Greens servitude Trafficking Protocol forced marriage Population rural and regional alcohol entitlements ministries Hung Parliament social citizenship maritime Iran ANZUS regional students school chaplains federal budget 2011-12 salary Medicare Locals primary care Building the Education Revolution Bills anti-corruption fraud bribery corporate ownership whistleblower G20 economic reform innovation Members of Parliament Scottish referendum early childhood education Middle East; national security; terrorism social services Criminal Code Amendment (Misrepresentation of Age to a Minor) Bill 2013 online grooming sexual assault of minors ACT Assembly national security smoking plain packaging tobacco cigarettes Asia; Japan; international relations Work Health and Safety Migration; asylum seekers; regional processing China; United States; international relations fiscal policy Racial Discrimination Act; social policy; human rights; indigenous Australians Foreign policy Southeast Asia Israel Palestine asylum refugees immigration political finance donations foreign aid disability employment Economics efficiency human rights; Racial Discrimination Act employment law bullying asylum seekers Animal law; food copyright Australian Law Reform Commission industry peace keeping contracts workplace policies same-sex marriage disorderly conduct retirement Parliament House standing orders prime ministers election timetable sitting days First speech defence budget submarines workers financial sector Canada Somalia United Kingdom GDP Tasmania world heritage political engagement leave loading Trade; tariffs; safeguards; Anti-dumping public interest disclosure whistleblowing Productivity Commission limitation period universities Ireland cancer gene patents genetic testing suspension of standing and sessional orders live exports infant mortality honorary citizen railways disciplinary tribunals standard of proof World Health Organisation arts international students skilled graduate visas temporary employment visas apologies roads Italy national heritage NHMRC nutrition anti-dumping Rent Assistance obesity evidence law sacrament of confession US presidential election international days DFAT UN General Assembly deregulation Regulation Impact Statements administrative law small business Breaker Morant regional engagement social determinants of health abortion Members suspension workplace health and safety marine reserves hearing TAFE Victoria astronomy resources sector YMCA youth parliament Korea fuel rebate Australian Greens presidential nomination Racial Discrimination Act political parties preselection solar hot water Financial Action Taskforce Horn of Africa peacekeeping piracy Great Barrier Reef Stronger futures political financing political education Social Inclusion Board early childhood National Quality Framework for Early Childhood Education and Care Murray-Darling Basin sanctions Norway hospitals republic President Barack Obama Presidential visits qantas counselling Korean peninsula Work Choices biosecurity hendra environmental law federalism federation preselection therapeutic goods Therapeutic Goods Administration plebiscites computer games pests suicide nuclear COAG Ministerial Councils floods ADHD stimulant medication advertising electricity extradition conscience votes poverty preventative health rural health coastal erosion Parliamentary Budget Office work-life balance

Show all
Show less
Back to top