预测-然后-优化任务之间的正确距离概念是什么?
What is the Right Notion of Distance between Predict-then-Optimize Tasks?
AI总结:
针对预测-然后-优化(PtO)框架下传统数据集距离无法有效衡量任务相似性的问题,本研究提出融合下游决策信息的OTD³距离,推导了PtO适配边界,实验验证其能准确预测模型在三类PtO任务上的可迁移性。
摘要:
比较数据集是机器学习中的一项基础任务,对多种学习范式至关重要——从评估训练与测试数据集以衡量模型泛化能力,到利用数据集相似性检测数据漂移。尽管传统的数据集距离概念提供了有理论依据的相似性度量,但其效用大多通过预测误差最小化来评估。然而,在预测-然后-优化(Predict-then-Optimize, PtO)框架中,预测结果作为下游优化任务的输入,模型性能通过决策遗憾而非预测误差来衡量。本研究提出OTD³(最优传输决策感知数据集距离,Optimal Transport Decision-aware Dataset Distance),这是一种新型数据集距离,除特征和标签外还纳入了下游决策因素。我们证明传统的特征-标签距离在PtO场景下缺乏信息量,而OTD³能更有效地捕捉适配成功率。我们还基于该距离推导出了PtO专属的适配边界。实验结果表明,我们提出的距离能准确预测模型在文献中三类不同PtO任务上的可迁移性。代码可在https://github.com/paularodr/OTD3获取。
英文摘要:
Comparing datasets is a fundamental task in machine learning, essential for various learning paradigms-from evaluating train and test datasets for model generalization to using dataset similarity for detecting data drift. While traditional notions of dataset distances offer principled measures of similarity, their utility has largely been assessed through prediction error minimization. However, in Predict-then-Optimize (PtO) frameworks, where predictions serve as inputs for downstream optimization tasks, model performance is measured through decision regret rather than prediction error. In this work, we propose OTD$^3$ (Optimal Transport Decision-aware Dataset Distance), a novel dataset distance that incorporates downstream decisions in addition to features and labels. We show that traditional feature-label distances lack informativeness in PtO settings, while OTD$^3$ more effectively captures adaptation success. We also derive a PtO-specific adaptation bound based on this distance. Empirically, we show that our proposed distance accurately predicts model transferability across three different PtO tasks from the literature. The code is available at https://github.com/paularodr/OTD3.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。