Posted 25/10/2013 by Luke Buckmaster
Making decisions about complex public policy issues inevitably involves the assistance of experts. On occasion, however, experts in a given area disagree in their judgements.
In such cases, how can non-experts go about deciding which experts to believe?
Expert disagreement is one of the issues addressed in a recent Parliamentary Library research paper, Expertise and public policy: a conceptual guide.
As noted in the paper, expert disagreement poses a problem because by definition non-experts are not in a strong position to decide which of the experts' judgements is the most correct.
Using social expertise
The paper argues that the only way that non-experts are able to appraise expertise and expert claims is through the use of social expertise. This is expertise using social judgements that enable them to determine who to believe when they are not in a position to judge what to believe.
The paper discusses four different types of evidence that a non-expert might consider in order to establish that the word of one expert is more credible than that of their rival.
This can be thought of as a framework for using social expertise to evaluate expert claims. The question is, what sort of framework does it provide? How well do each of its elements stand up to scrutiny?
Which expert seems the more credible?
This refers to one expert being able to demonstrate ‘dialectical superiority’ over the other in, for example, a debate. The emphasis is on how statements are presented, rather than the content of the statements themselves. Examples include the ability to provide rebuttals, quickness or smoothness of response and more clearly explaining the evidence presented.
One advantage of this approach is that it allows non-experts to apply the kinds of social judgment they use in everyday life in assessing credibility claims. However, such evidence may simply be the result of better debating skills or ‘stylistic polish’. It is also doubtful that any marks of dialectical superiority are universal. Rather, it seems more likely that they are dependent on subject matter and the context in which the arguments are presented.
Who has the numbers on their side?
A common strategy for justifying acceptance of one expert’s conclusions over a rival’s is that more experts agree with the former than the latter, for example, the argument that there is an overwhelming consensus among climate scientists that human activity is having an impact on the climate.
Arguably, non-experts are relatively well placed to employ the strategy of going by the numbers—understood as being able to ‘read the scientific consensus’ on a particular controversy.
A practical problem with this strategy is that a scientific consensus may not be available (the science may not yet be 'in'). Also, some question this approach on the grounds that experts on one side of an argument may not be sufficiently independent from one another. While others have suggested that this problem has been overstated, it is fair to say that 'going by the numbers' is an approach that is most justifiably used in cases where the numbers are substantially in favour of one side.
Are there any relevant interests or biases?
Another strategy in deciding between conflicting expert claims is evidence of distorting interest and biases (for example, pecuniary interests).Interests are ‘often one of the more accessible pieces of information that a novice can glean about an expert’.
There are two main objections to the conflict of interest approach.
First, we can't simply assume that interest will distort conclusions. Second, interest objections add little to attempts to evaluate the validity of an expert argument. The ability to demonstrate the distorting impact of a conflict of interest will be on the basis of the quality of the arguments raised, meaning one does not need the interest-objection.
At best, interest based arguments should probably be used as a prompt for closer examination of an expert’s claim.
What are the experts track records?
For non-experts, information about whether an expert has gotten things right in the past can provide insight into an expert's credibility.
However, instances in which a non-expert will have access to the track records of competing experts are not that common. Also, in some cases the ‘true’ outcome of an expert dispute can itself be highly contested. Further, a non-expert would be unwise to assume that an expert with a good track record could automatically be trusted on all matters into the future.
An ethic of social expertise?
By identifying the above strategies’ strengths and limitations, the paper suggests how each might best be used.
First, they can be used in combination to improve their strength and reliability. Second, non-experts are able to draw on advice from meta-experts(experts on experts), independent non-partisan expert bodies, and less formally, policy blogs focused on communicating technical policy areas to a non-expert audience.
The important point is that because non-experts possess social expertise they are potentially able to make reasoned judgements about expert claims on technical matters. This means that they are not necessarily passive recipients of expert activity.
However, if non-experts are to become more active in their use of experts and meta-experts (developing what might be called an ethic of social expertise), then they should strive to better understand expertise and be more systematic and considered in their use of social expertise.
*Co-authored with Matthew Thomas
Image source: Wikimedia Commons