06.7.2 · Skill in Evidence-based decisions
Causal reasoning
Telling correlation from causation in product data.
Causal reasoning asks what else could explain a pattern. When experiments are not possible, methods like difference-in-differences or natural experiments help. Most bad product decisions from data come from treating correlation as cause.
Resources
3- articleGo deeperHow and when to control for confounders during product usage analyses ↗Heap · 2021 · 10 min
Uses the familiar "users of this feature retain better" case to show how a confounder fakes an effect and how to adjust for it.
- bookGo deeperDifference-in-differences ↗Matheus Facure · 25 min
The method to reach for when a change shipped to one group or market and you cannot run an A/B test.
- bookGo deeperCausal inference for the brave and true ↗Matheus Facure · free online book
A free, rigorous and readable book on causal inference with Python code, from randomized tests to methods for when you cannot run one.
Tools
none yetTools for this topic are being researched.