The expertise problem
Our ability to make smart decisions hinges on our knowledge of the subject we make decisions about. We can substitute our own knowledge by other experts, but how do we know the quality of other experts if we do not understand the subject matter ourselves?
Expertise is not only the knowledge of, or experience with, a specialist craft. Having all the knowledge in the world does no good if it is not acted upon.
Being an expert means to provide reliable knowledge for practical purposes, and needs to complement the decision-making process for stakeholders.
Corollary: Your value as an expert is your ability to act and show results from your specialist knowledge. Only communicating the knowledge itself is worthless as a professionel expert or scientist—that is the role of a teacher.
If that is our definition of an expert; how can a decision maker, when choosing a new expert to trust, recognize a priori expertise? Obviously, this ignores the chicken or the egg dilemma, but in our current knowledge-heavy society experts should be aplenty.
Trusting the wrong expert can certainly be fateful.
Some academic work has gone into expertise finding, but as we can see from its wikipedia article; it is still a poorly understood concept.
Strategies for choosing an expert
I see five different strategies for choosing an expert.
Using the right strategy depends on the context.
1. Trust in authority
The strategy that requires the least amount of cognitive shifting is trusting an authority.
When we put our trust an authority, our choice is only whether or not to use the recommendation. This short circuits the normal use of critical thinking when making decisions by the other strategies.
A study from 2010 shows that people with experience in hectic and stressful environments (first person shooter video games) have a higher amount of cognitive flexibility.
This tells us that either 1) people with a higher amount of cognitive flexibility find themselves in these environments, or that 2) these environments foster the development of cognitive flexibility. Whether 1 and/or 2 is true is not important, but it shows that there exists a pressure in these environments where a certain amount of cognitive flexibility is needed to perform. Because we do not have endless amounts of cognitive flexibility, it is a good idea to prefer strategies that require the least amount of cognitive flexibility.
The problems with trusting an authority include:
- Their values and priorities might now align with ours
- Their competencies might not be good enough
- Their intentions might not be in our best interest
A typical example of choosing an expert by trusting an authority is getting a public defender in the court of law.
2. Trust in peer review
If we have to find a contractor (expert) to renovate our house, we might try to find all the available information about the contractors we are considering for hire, including public reviews and implications in court cases. All available information is information put forth, by other people i.e. peers, which we review ourselves.
The amount of work/time/cognitive shifting needed to choose an expert by this strategy is significantly higher than when using an authority, since one have to critically weigh different, and possibly contradicting, facts and opinions from many different sources.
Choosing an expert by trust in peer review conceptually looks like choosing an expert via trust in an authority, if we think about our peers as an authority and summarize peer critique into rating systems. In practice, however, they look nothing alike if we qualitatively make use of our peers, since critical thinking is then employed to assess the gathered data.
The peer review process does not suffer the same drawbacks as trusting an authority, but unfortunately it has another drawback, as mentioned earlier; the peer review process needs significantly more time to utilize and requires more cognitive shifting.
3. Navigating the chaos by instinct
Some people sometimes find themselves in a hectic environment with limited ressources and no one with higher authority to rely on. This could be a military officer on an active battlefield, where the situation literally is of life-or-death importance.
This officer has to make decisions and delegate responsibility and tasks to their subordinates, and choosing the wrong specialist for some task will ensure failure for all. Hopefully there are no bad specialist choices on the battlefield, but the stakes cannot get higher than this.
Using the strategy with the lowest amount of cognitive shifting is important, and something most military forces recognize. Military officers are extensively trained in “tactics” to help minimize cognitive shifting once shit hits the fan and important decisions have to be made.
Unfortunately, no higher authority is available to help military officers with decision making in many situations once they are deployed, which means that choosing an expert by trusting an authority is not an option.
“Instinctual leadership” is a controversial topic due many of its proponents, that shows no proper academic rigor, but uses the concept as a marketing ploy for personal gain1234.
While many decisions made on the basis of instinct alone can be considered suboptimal in hindsight, it is important to note that it is a valid strategy for certain situations.
4. Having enough insight to evaluate expertise
Some people make their living evaluating other people’s skill and culture fit.
When going through a hiring process for a company, normally, one would go through an unstructured interview, even though a lot of research have shown how counterproductive that is.
How good we are at evaluating our own skills and abilities is not well understood. Evaluating other people’s skills and abilities is hard, when we do not know how good we are ourselves. A good skeptic would therefore need to be highly aware of how they evaluate other’s expertise.
Being an expert should make us able to evaluate other experts within our speciality—we just need to be sure to keep our understanding of the world up-to-date so we do not fall behind!
Ain’t nobody got time for that!
Where is the demarcation line of being a non-expert, non-expert with enough insight to evaluate other experts and just being the expert? I do not know. If you know, please tell me. Choosing an expert by trusting peer review seems like a good choice suddenly…
5. Be the expert
By definition, if you are an expert, your expertise will include the abilitiy to evaluate other people’s work, and therefore other people’s potential expertise.