Retail intelligence often relies on tracking popular, high‑velocity items to infer macro‑economic indicators, which can ignore the “long tail” of niche products and introduce selection bias. This paper conducts a simulation study to evaluate selection bias in inflation estimation and compares Inverse Probability Weighting (IPW) with stratification across diverse data‑generating processes.
The experiment consists of 400 Monte Carlo replications covering four scenarios:
- Aligned step functions
- Smooth gradients
- Misaligned breaks
- Polynomial relationships
For each scenario we test five IPW specifications and stratification with varying numbers of strata. Key findings:
- Stratification outperforms IPW in three scenarios, keeping median error below 0.04 percentage points even when strata boundaries are deliberately misaligned with population breaks (≈116× advantage over IPW).
- When the true relationship is a smooth polynomial, IPW with a spline propensity model achieves the lowest median error (0.007pp) versus stratification’s 0.013pp, highlighting context‑dependency.
- An “oracle” IPW with perfect structural knowledge still incurs a 6.06pp error in step‑function scenarios, while stratification remains at 0.008pp. This reflects a violation of the Positivity Assumption—selection probabilities can differ dramatically (e.g., 90% vs 1%), pushing weighting methods outside their theoretical design envelope.
Conclusion: In retail long‑tail distributions with severe positivity violations, stratification offers a safer and more robust engineering choice.
Blogger's Review: The study serves as a cautionary tale against the blind application of IPW in real‑world retail data, where uneven selection probabilities are common. Simple stratification, despite its apparent naivety, consistently delivers reliable estimates and should be the default tool for practitioners.