arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.19667cs.LGcs.AI
收藏

协调下游决策中的模型多重性

Reconciling Model Multiplicity for Downstream Decision Making

Carnegie Mellon University(卡内基梅隆大学);University of Minnesota(明尼苏达大学)

Ally Yalei Du, Dung Daniel Ngo, Zhiwei Steven Wu

AI总结:

针对下游决策中精度相同但最佳响应行动不一致的模型多重性问题,提出基于多重校准的框架协调预测差异,使模型在决策者视角下与真实分布不可区分,实验证明其能显著降低决策损失并提升行动一致性。

摘要:

我们考虑下游决策中的模型多重性问题,即两个具有同等精度的预测模型无法就下游损失函数的最佳响应行动达成一致的设定。我们表明,即使两个预测模型在几乎所有地方都几乎一致地同意其各自的预测,它们诱导的最佳响应行动仍有可能在大部分群体上存在差异。我们通过提出一个框架来解决这一问题,该框架同时针对下游决策问题和个体概率预测对预测模型进行校准。具体而言,利用多重校准工具,我们提供了一种算法,在每个时间步,首先协调个体概率预测中的差异,然后校准更新后的模型,使得决策者无法将它们与真实概率分布区分开来。我们将结果扩展到无法直接访问真实概率分布,而是依赖一组独立同分布数据作为经验分布的设定。最后,我们提供了一组实验来实证评估我们的方法:与现有工作相比,我们提出的算法生成了一对预测模型,不仅改善了下游决策损失,而且在几乎所有地方都就最佳响应行动达成一致。

英文摘要:

We consider the problem of model multiplicity in downstream decision-making, a setting where two predictive models of equivalent accuracy cannot agree on the best-response action for a downstream loss function. We show that even when the two predictive models approximately agree on their individual predictions almost everywhere, it is still possible for their induced best-response actions to differ on a substantial portion of the population. We address this issue by proposing a framework that calibrates the predictive models with regard to both the downstream decision-making problem and the individual probability prediction. Specifically, leveraging tools from multi-calibration, we provide an algorithm that, at each time-step, first reconciles the differences in individual probability prediction, then calibrates the updated models such that they are indistinguishable from the true probability distribution to the decision-maker. We extend our results to the setting where one does not have direct access to the true probability distribution and instead relies on a set of i.i.d data to be the empirical distribution. Finally, we provide a set of experiments to empirically evaluate our methods: compared to existing work, our proposed algorithm creates a pair of predictive models with both improved downstream decision-making losses and agrees on their best-response actions almost everywhere.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑