arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.17769cs.IRstat.MEstat.ML
收藏

两阶段风险控制及其在排序检索中的应用

Two-stage Risk Control with Application to Ranked Retrieval

Yunpeng Xu, Mufang Ying, Wenge Guo, Zhi Wei

AI总结:

针对多阶段机器学习系统的风险控制难题,本文基于 LTT 和 CRC 框架提出利用问题顺序特性的两阶段风险控制方法,设计了新型损失函数,并在 MSLR-Web 和 Yahoo LTRC 数据集上验证了其有效性与理论保证。

摘要:

实际的机器学习系统通常在多个连续阶段运行,如排序和推荐系统,一般包括检索阶段和随后的排序阶段。由于其固有的复杂性,有效评估预测不确定性并确保此类系统中的有效风险控制带来了重大挑战。为应对这些挑战,我们基于近期提出的先学习后测试(LTT)和共形风险控制(CRC)框架,开发了两阶段风险控制方法。与先前处理多种风险的方法不同,我们的方法利用了问题的顺序特性,从而降低了计算负担。我们为所提方法提供了理论保证,并设计了专为排序检索任务定制的新型损失函数。通过在两个大规模、广泛使用的数据集 MSLR-Web 和 Yahoo LTRC 上的实验,验证了我们方法的有效性。

英文摘要:

Practical machine learning systems often operate in multiple sequential stages, as seen in ranking and recommendation systems, which typically include a retrieval phase followed by a ranking phase. Effectively assessing prediction uncertainty and ensuring effective risk control in such systems pose significant challenges due to their inherent complexity. To address these challenges, we developed two-stage risk control methods based on the recently proposed learn-then-test (LTT) and conformal risk control (CRC) frameworks. Unlike the methods in prior work that address multiple risks, our approach leverages the sequential nature of the problem, resulting in reduced computational burden. We provide theoretical guarantees for our proposed methods and design novel loss functions tailored for ranked retrieval tasks. The effectiveness of our approach is validated through experiments on two large-scale, widely-used datasets: MSLR-Web and Yahoo LTRC.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑