...
There are three main categories of biases to be aware of when conducting analysis: Selection, Social and Process. Reflect on the types of biases introduced in the process of analysis and try to mitigate them as much as possible (see tips below the description of each biasebias). Biases should be described in the methodology of the output or report.
Bias | Description | Examples |
|---|
| Selection bias | caused by choosing non-random data for analysis. Some information is unconsciously chosen or disregarded, misleading the analyst into a wrong conclusion. | - Absence of evidence
- Anchoring Effect
- Availability Cascade
- Confirmation Bias
- Conservatism Bias
- Satisficing or Premature Closure
| - Evidence Acceptance
- Pro-Innovation Bias
- Publication Bias
- Recency
- Salience or Vividness
- Survivorship
| Mitigation techniques |
| The key to overcoming selection biases is to examine carefully the credibility and reliability of sources and data use to base the analysis. |
Usability of Data - Are they relevant to your research topic?
- Are they complete?
- Are they sufficiently recent?
- Are they sensitive?
- Are they representative?
- Are they comparable to other data you have available?
- Are they trustworthy?
| Credibility of the Data: - Evaluate how accurate and precise the information is
- Check for strong corroboration and consistency with other sources
- Look for negative cases
- Identify key themes indicated in the evidence
- Consider if the explanation is plausible given the context
- Re-examine previously dismissed information or evidence
- Consider whether ambiguous information has been interpreted and caveated properly
- Indicate the level of confidence in references
| Reliability of Sources - Qualifications and technical expertise of the source
- Reputation and track record for accuracy
- Its objectivity and motive for bias
- It's proximity to the original source or event
|
| Bias | Description | Examples |
| Social Bias | the result of interactions with other people. The way we are processing and analyzing information depends on our relations with the persons who provided us with the information or hypothesis. | - Institutional Bias
- Halo Effect
- Stereotyping
- Implicit Association
| - Attribution Error
- False Consensus
- Group Think
- Mirror Imaging (or Projection)
| Institutional Bias | Halo EffectStereotypingImplicit Association
| Mitigation techniques | The key to overcome social biases is to examine carefully the number of assumptions used to fill information gaps and to actively seek alternative hypothesis. |
- Alternate Hypothesis
- Competing Hypothesis
- Devil’s Advocacy
- Differential Diagnosis
- Key Assumptions Checklist
- Logic Mapping
| - Challenging a view or consensus by building the best possible case for an alternative explanation and explicitly contesting key assumption to see if they will hold.
- Identify key assumptions
- Selection one or more assumptions that seem susceptible to challenge
- Review the evidence to determine if some are questionable validity
- Highlight any evidence that could support an alternative hypothesis or contracts the current thinking
|
Process | our tendency to process information based on cognitive factors rather than evidence. When we process information, we often display inherent thinking errors. They prevent an analyst from accurately understanding reality even when all the needed data and evidence are in his/her hands. | - Blind Spot
- Overconfidence
- Choice-Supportive
- Clustering Illusion
- Hindsight Bias
- Irrational (Commitment) Escalation
- Selective Attention/Perception
- Information Volume Bias
- Framing
| - Hyperbolic Discounting
- Impact
- Negativity
- Ostrich Effect
- Planning Bias
- Status Quo
- Wishful Thinking
- Risk-averse
- Zero Risk
|
| Bias | Description | Examples |
|
| Mitigation techniques | By making the way the information processed obvious to everyone, members of a team can acknowledge the limitations and advantages of each of the roles. |
| - Edward de Bono’s method is a parallel thinking process that helps analysts overcoming their assumption, biases, and heuristics.
- Members of a team are assigned with a “role” to play a hat to wear.
- Members of a team are assigned with a role to play, a hat to wear
- They can more easily examine a hypothesis from different angles: neutral, emotional, creative, optimist, and pessimist angle.
 Image Modified | - Structured analytical thinking can be used to overcome cognitive limitations and develop analytical objectivity.
 Image Modified | |
Good skills, attitudes, and habits for anyone involved in an analysis process
...