Abstract
User retention is a key focus for consumer based internet companies and promotions are an effective lever to improve retention. However, companies rely either on non-causal churn prediction to capture heterogeneity or on regular A/B testing to capture average treatment effect. In this paper, we propose a heterogeneous treatment effect optimization framework to capture both heterogeneity and causal effect. We propose two algorithms to maximize heterogeneous treatment effect: 1) Ranking based on point estimates of heterogeneous treatment effects obtained using existing esti- mation methods with training labels adjusted based on Lagrangian Subgradient method. 2) A novel ranking algorithm which combines estimation and optimization in one stage and directly optimizes for the aggregated targeted treatment effect. We also develop an evaluation metric that captures the real-world business value of different methods and use this to evaluate various approaches on our large-scale experiment data set both offline and online. Our algorithm (approach 2) performs significantly better than random explore benchmark and existing estimators (approach 1) in both offline and online tests. This method has been deployed to production and is currently live in multiple cities all over the world.
Authors
Shuyang Du, James Lee, Farzin Ghaffarizadeh
Conference
KDD 2019
Full Paper
Uber AI
