Confirmation Bias Countermeasures

Confirmation bias is the tendency to seek, interpret, and remember information that confirms your existing hypothesis while ignoring information that contradicts it. In research, this manifests as unconsciously weighting supporting evidence more heavily than contradicting evidence, stopping research once you have "enough" support for your hypothesis, and interpreting ambiguous data in the direction of your prior belief. Every analyst has confirmation bias. The difference between amateur and professional is whether you build countermeasures.

1. Pre-Register Your Hypothesis Before starting research, write down your hypothesis and the specific evidence that would confirm or refute it. This prevents post-hoc rationalization — adjusting the hypothesis to match the evidence you found instead of evaluating the evidence against the hypothesis you started with.
2. Mandatory Disconfirmation Search After gathering supporting evidence, explicitly search for contradicting evidence with the same rigor. Spend equal time looking for evidence that your hypothesis is wrong. If you cannot find any, your hypothesis is either correct or your search was not thorough enough.
3. Adversarial Review Have your analysis reviewed by someone who disagrees with your hypothesis — or use the red team prompting technique to generate counterarguments systematically. An analysis that survives adversarial review has earned its confidence level.