arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.01412cs.LGstat.ME
收藏

面向精准因果效应估计的超参数调优挑战

The Challenges of Hyperparameter Tuning for Accurate Causal Effect Estimation

School of Engineering, CHAI Hub University of Edinburgh(爱丁堡大学工程学院、CHAI Hub大学);Institute for Analytics and Data Science University of Essex(埃塞克斯大学分析与数据科学研究所);Institute for Social and Economic Research University of Essex(埃塞克斯大学社会科学与经济研究所)

Damian Machlanski, Spyridon Samothrakis, Paul Clarke

AI总结:

针对因果推断任务缺乏统一超参数调优指标、模型选择复杂的问题,通过多基准数据集的实证研究,验证了超参数调优对提升因果效应估计前沿性能概率的作用,并指出标准指标表现不一致,需探索稳定的通用评估指标。

摘要:

机器学习(ML)在基于观测数据估计干预对结果的因果效应方面发挥着日益关键的作用。针对该任务,学界已提出众多机器学习方法(即“因果估计器”)。与所有机器学习方法一样,这些方法全部需要大量的超参数调优。对于非因果预测任务,调优指标的选择已形成共识(如均方误差),使得模型对比十分简便。但在因果推断任务中,这类共识尚未达成,导致因果模型的对比难度很大。除此之外,目前不存在可用于调优因果估计器的理想指标,因此研究者必须依赖代理指标。进一步而言,因果推断中的模型选择涉及多个组成部分(因果估计器、机器学习回归器、超参数、指标),这让问题变得更加复杂。为评估每个组成部分的重要性,我们针对其组合开展了大规模实证研究。我们的实验设置涵盖多种常用因果估计器、回归器(下文称“基学习器”)以及指标,应用于四个知名的因果推断基准数据集。结果显示,仅使用常用估计器时,超参数调优将达到平均因果效应估计(65%→81%)和个体化因果效应估计(50%→57%)前沿性能的概率有所提升。我们还发现,标准指标的表现在不同场景下可能存在不一致性。本研究结果凸显了进一步研究的必要性:需探明是否能找到可在因果模型评估中稳定实现前沿性能的指标。

英文摘要:

ML is playing an increasingly crucial role in estimating causal effects of treatments on outcomes from observational data. Many ML methods (`causal estimators') have been proposed for this task. All of these methods, as with any ML approach, require extensive hyperparameter tuning. For non-causal predictive tasks, there is a consensus on the choice of tuning metrics (e.g. mean squared error), making it simple to compare models. However, for causal inference tasks, such a consensus is yet to be reached, making any comparison of causal models difficult. On top of that, there is no ideal metric on which to tune causal estimators, so one must rely on proxies. Furthermore, the fact that model selection in causal inference involves multiple components (causal estimator, ML regressor, hyperparameters, metric), complicates the issue even further. In order to evaluate the importance of each component, we perform an extensive empirical study on their combination. Our experimental setup involves many commonly used causal estimators, regressors (`base learners' henceforth) and metrics applied to four well-known causal inference benchmark datasets. Our results show that hyperparameter tuning increased the probability of reaching state-of-the-art performance in average ($65\% {\rightarrow} 81\%$) and individualised ($50\% {\rightarrow} 57\%$) effect estimation with only commonly used estimators. We also show that the performance of standard metrics can be inconsistent across different scenarios. Our findings highlight the need for further research to establish whether metrics uniformly capable of state-of-the-art performance in causal model evaluation can be found.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑