Rational Discussion of Controversial Topics
Economics Education Ethics Foreign Policy Government History Politics Religion Science

When can experts be trusted?

melian          30 May 2015 06:22 PM

In the XVth century one could become an expert in a dozen of completely different areas and even contribute to their advancement. Today even the most brilliant scientists are familiar with just a small part of their own field. The fraction of the total human knowledge that can be mastered by any individual grows exponentially smaller and, inevitably, we become more and more dependent on experts.

Unlike, for example, our growing dependence on inanimate technology, the dependence on other humans creates trust-related issues. How can a non-expert distinguish a real expert from a fake one?

The options I can think of are:
  • Relying on the wisdom of the crowd, which basically means accepting as experts those whom the majority believes to be experts. Though better than the blind trust, this method is still rather unreliable.

  • Taking into account only the opinions of educated people. While this might be a step forward, history shows that this method is far from being foolproof.

    Take, for example, the Great Plague of the XIVth century which killed about quarter of Europe. During the plague, thousands of ignorant people fled towns in the attempt to escape the wrath of God (a popular explanation for the calamity). In the meanwhile, educated skeptics observed their flight with derision, having concluded, with impeccable logic, that hiding from the Almighty is impossible.

    It would be wrong to put all blame for this fatal mistake on religion. The XVIIIth century Enlightenment greatly increased the prestige of scientific reasoning compared to theological speculations. Yet, in the field of medicine, ignorance was still often safer than knowledge. Until at least the middle of the XIXth century, a sick person may have been safer when treated by potions of self-appointed village witches than by the university trained doctors with their attachment to bloodletting.

  • Personally, I prefer whenever possible to use the following heuristic. I check whether the supposed experts can demonstrate an achievement which is obvious even to non-experts. For example, most people don’t understand the Maxwell equations. Yet, the fact that with one push of a button they can turn their TVs on demonstrates that physics is a real thing. Thus, when physicists claim the existence of electrons, they deserve more trust than mediums claiming the existence of spirits. Similarly, the drastic decline in human mortality rates demonstrates that medicine has changed a lot since the times when humorism was the standard paradigm.

A more challenging problem is to decide when to trust the experts. The only useful heuristics I know are:
  • Checking for conflicts of interest (unfortunately, they arise almost everywhere).

  • Checking if the experts are unanimous (“asking for a second opinion”)

  • Checking if the question truly falls into their area of expertise.

Do you know other useful methods for addressing these two problems? Which methods do you use when forming an opinion on issues like GW or the validity of social sciences?

Would you like to read similar articles in the future?
-2 -1 0 +1 +2

                  Post Comment                   

Recommended for You Optimates Populares Centrists

Show comments            Sort by        

melian 4 June 2015 05:30 AM

This may be relevant.


Dahlen 10 June 2015 09:07 AM

I don't find it necessary or proper for me to form an opinion on everything; I only become invested in a position when it's something that demands my attention by impacting me directly (e.g. controversies about some health problem I might have), or when I intrinsically like researching it. Otherwise, I shrug and assume the scientific establishment is right, and possibly make low-effort adjustments to my lifestyle or behaviour (e.g. try to walk and bike more and eat less meat to lower my carbon footprint, or let sociology studies influence my opinion, for your examples).

BTW, why global warming and social sciences? The only people I've seen to reject these two as scientifically invalid, especially in conjunction, were strongly partisan.


ChristianKl 3 June 2015 07:44 AM

Trust depends on context. If you make a decision you have to consider the costs of false positives and false negatives.

When a drug company wants to get a drug approved we have a high standards for the drug company proving benefits of the drug.
We have a lower bar of evidence for side effects.
We don't need a placebo-blind trial to identify side effects to write on the back of a drug box but we do require the placebo-blind trial for benefits of the drug.

It's not useful to give blind trust to doctors because you think that medicine has increased lifespan. Even if the Rand study would have found that medicine works.
Even if a study says that the average person benefits from a drug you aren't the average person. It's important to actually check whether the drugs you are taking are having a positive effect for you.

You might even use a marker to mark the right leg when going to a leg operation, as suggested on LW:
Avoidable medical errors might be the second leading cause of death after CVD.[1] This makes a hospital visit possibly the most dangerous thing you can do, especially if you are young. In general, you should not assume that medical staff are competent. Triple check dangerous prescriptions. If you don’t know whether a prescription is dangerous, assume it is. Ask medical staff if they’ve washed their hands (yes, this is actually still a major problem). Sharpie on yourself which side of your body a surgery is supposed to happen on, along with your name and what the surgery is for


petergast 29 October 2015 10:26 PM

I like your heuristic. I expand it slightly:

1. Can they predict the future?

For example Darwin predicted that fossils of forms intermediate between man and apes would be found, probably in Africa.

On the other hand, psychologists and psychiatrists seem to be very bad at predicting people's behavior - worse than simple machine learning programs.

2. Can they design and / or make things that work impressively?

Your example of physics. On the other hand, macro-economists' predictions seem to be pretty worthless.

3. Can they explain things accurately, and with fewer moving parts than others?

Micro-economics does well here. Theology does badly e.g. it predicts that sincere repentance would be followed by better behavior, which doesn't seem to work very well eg with pedophile priests.

From applying this I find vast classes of 'experts' who are not worth listening to, other than for entertainment.


FrameBenignly 1 June 2015 02:57 AM

The question isn't how likely are the experts to be right. It's are experts more likely to be right than your wild guess. Even if their probability of being correct is quite low; it still should be treated as more accurate than your own opinion.

There is a flaw when they have a clear source of bias which you lack such as their funding sources. Even when no such conflicts exist I'd say people are more willing to ignore experts than to listen to them when they should, and to find a minor excuse.

An important thing to distinguish between is the expert's opinion and the general consensus in their field. The general consensus should be treated with a great deal of respect. Experts may say their personal opinion sometimes for an area that is still highly uncertain, and you should verify their certainty.


VoiceOfRa 30 May 2015 11:01 PM

The problem with relying on Expert unanimity is that its subject to Goodhart's law.


ChristianKl 3 June 2015 07:44 AM

When thinking about global warming you can go and read mainstream news articles that quote climate scientists to form your opinion of the position of climate scientists.
You can also read the IPCC summary for policy makers. The IPCC document has much higher quality.
It's has confidence values for various predictions surrounding global warming by scientists who went through credence training. It's much more nuanced than the view you
get if you simply get your views from the mainstream media.

If you focus on the core global warming question, there are two things you can be doubtful about.
Is the IPCC confidence interval large enough? Is the uncerainity higher than they claim it is?
Is the average temperature increase prediction of the IPCC right or should we use a different average?

I think as Nassim Taleb that the uncertainty is likely higher than the IPCC numbers. On the other hand I see no
reasonable argument for a lower average temperature increase than what climate scientists predict.


VoiceOfRa 2 June 2015 10:48 PM

I like Taleb's approach to this problem:

Does the "expert" have skin in the game?


Fwiffo 2 June 2015 12:36 PM

If you are not an expert expert you can't know who knows what. However people don't usually have to know that a person can be trusted to actually trust them. It could also be argued that since "proof of trusthworthiness" is a rare thing it is not practical to keep trust to that standard. People in fact trust experst even when they "can't" be trusted. Having had a good faith educaiton to be trained as an expert in that field is a thing you can measure even if it doesn't 100% correlate with ability.

Personally I can live with occasionally having to doubt expert opinion. It helps that for most thinds I dont' need 100% reliability and often if there is something a miss it's more of a question when it is found out rather than if it is found out. I am answering never.