Machine learning (ML) estimates of conditional average treatment effects (CATE) can guide policy decisions, either by allowing targeting of individuals with beneficial CATE estimates, or as inputs to decision trees that optimise overall outcomes. There is limited information available regarding how well these algorithms perform in real-world policy evaluation scenarios. Using synthetic data, we compare the finite sample performance of different policy learning algorithms, machine learning techniques employed during their learning phases, and methods for presenting estimated policy values. For each algorithm, we assess the resulting treatment allocation by measuring deviation from the ideal ("oracle") policy. Our main finding is that policy trees based on estimated CATEs outperform trees learned from doubly-robust scores. Across settings, Causal Forests and the Normalised Double-Robust Learner perform consistently well, while Bayesian Additive Regression Trees perform poorly. These methods are then applied to a case study targeting optimal allocation of subsidised health insurance, with the goal of reducing infant mortality in Indonesia.
Wasserstein Policy Learning for Distributional Outcomes
School of Computing and Information Technology, Great Bay University(广东大湾大学计算机与信息科技学院);Department of Data Science, City University of Hong Kong(香港城市大学数据科学系);School of Statistics and Data Science & Institute of Big Data Research, Shanghai University of Finance and Economics(上海财经大学统计与数据科学学院及大数据研究院)
Contextual Bandits in a Survey Experiment on Charitable Giving: Within-Experiment Outcomes versus Policy Learning
Stanford Graduate School of Business(斯坦福大学商学院);Amazon(亚马逊);Stanford Management Science and Engineering(斯坦福大学管理科学与工程系);Coupang(库胖);University of Toronto Computer Science(多伦多大学计算机科学系)