增益来源:条件平均剂量反应估计中的性能分解
Sources of Gain: Decomposing Performance in Conditional Average Dose Response Estimation
AI总结:
针对现有CADR估计器基准评估的不足,提出性能分解方案,经近1500次实验发现基准挑战与宣称不符,混杂因素并非核心问题,为后续研究指明方向。
摘要:
估计条件平均剂量反应(CADR)是一个重要但具有挑战性的问题。估计器必须正确建模协变量、干预措施、剂量与结果之间可能存在的复杂关系。近年来,机器学习界对开发针对特定挑战的定制化CADR估计器表现出极大兴趣,其性能通常在(半)合成基准数据集上与其他方法对比评估。本文分析了这一做法,表明仅使用流行基准数据集而不做进一步分析不足以评判模型性能。现有基准包含多重挑战,必须厘清各自的影响。因此,我们提出一种新颖的分解方案,可评估五个不同组成部分对CADR估计器性能的影响。我们在四个广泛使用的基准数据集上对八种流行CADR估计器应用该方案,开展了近1500次独立实验。结果显示,大多数现有基准具有挑战性的原因与其创建者的主张不同。值得注意的是,作为大多数估计器应对的核心挑战,混杂因素在所考虑的任何数据集中都不构成问题。我们讨论了这些发现的主要意义,并提出了未来研究方向。
英文摘要:
Estimating conditional average dose responses (CADR) is an important but challenging problem. Estimators must correctly model the potentially complex relationships between covariates, interventions, doses, and outcomes. In recent years, the machine learning community has shown great interest in developing tailored CADR estimators that target specific challenges. Their performance is typically evaluated against other methods on (semi-) synthetic benchmark datasets. Our paper analyses this practice and shows that using popular benchmark datasets without further analysis is insufficient to judge model performance. Established benchmarks entail multiple challenges, whose impacts must be disentangled. Therefore, we propose a novel decomposition scheme that allows the evaluation of the impact of five distinct components contributing to CADR estimator performance. We apply this scheme to eight popular CADR estimators on four widely-used benchmark datasets, running nearly 1,500 individual experiments. Our results reveal that most established benchmarks are challenging for reasons different from their creators' claims. Notably, confounding, the key challenge tackled by most estimators, is not an issue in any of the considered datasets. We discuss the major implications of our findings and present directions for future research.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。