在估算CATE方面做得很好?治疗效果估计量基准比较中被忽视的假设
Doing Great at Estimating CATE? On the Neglected Assumptions in Benchmark Comparisons of Treatment Effect Estimators
摘要:
用于从观察数据估计异质性治疗效果的机器学习工具箱正在迅速扩展,但其许多算法仅在非常有限的半合成基准数据集上进行了评估。在本文中,我们表明,即使在可以说是最简单的设置-估计下的可验证性假设-这种实证评估的结果可能是误导性的,如果(i)基准数据集的数据生成机制的假设和(ii)它们与基线算法的相互作用没有得到充分的讨论。我们详细考虑了两个流行的机器学习基准数据集,用于评估异质性治疗效果估计器-IHDP和ACIC 2016数据集。我们确定了它们当前使用的问题,并强调基准数据集的固有特征有利于某些算法-这一事实很少被承认,但对于解释经验结果具有巨大的相关性。最后,我们讨论了影响和可能的下一步措施。
英文摘要:
The machine learning toolbox for estimation of heterogeneous treatment effects from observational data is expanding rapidly, yet many of its algorithms have been evaluated only on a very limited set of semi-synthetic benchmark datasets. In this paper, we show that even in arguably the simplest setting -- estimation under ignorability assumptions -- the results of such empirical evaluations can be misleading if (i) the assumptions underlying the data-generating mechanisms in benchmark datasets and (ii) their interplay with baseline algorithms are inadequately discussed. We consider two popular machine learning benchmark datasets for evaluation of heterogeneous treatment effect estimators -- the IHDP and ACIC2016 datasets -- in detail. We identify problems with their current use and highlight that the inherent characteristics of the benchmark datasets favor some algorithms over others -- a fact that is rarely acknowledged but of immense relevance for interpretation of empirical results. We close by discussing implications and possible next steps.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。