arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.09414stat.MLcs.LG
收藏

超越增强动作代理的多专家学习延迟决策

Beyond Augmented-Action Surrogates for Multi-Expert Learning-to-Defer

School of Computing(计算学院);Fédération ENAC(ENAC联合会);ISAE-SUPAERO;ONERA;National University of Singapore(新加坡国立大学);Université de Toulouse(图卢兹大学);Agency for Science, Technology and Research(科技研究局);Institute for Infocomm Research(信息通信研究所)

Yannis Montreuil, Axel Carlier, Lai Xing Ng, Wei Tsang Ooi

AI总结:

针对多专家学习延迟决策问题,提出一种解耦代理损失函数,通过独立sigmoid头与softmax分类器头分离优化,解决了现有方法中的优化病理问题,并首次给出不随专家数量增长的校准常数界。

摘要:

学习延迟决策(L2D)系统针对每个输入决定是自行预测还是交给若干可用专家之一。非常成熟的方案通过将$K$个类别和$J$个专家视为共享$(K{+}J)$动作几何中的竞争动作,联合训练分类器和路由器。后续工作在该几何内提出了一系列增量修复;我们表明,即使在统计一致性下,每个方法仍不同程度地遭受优化层面的病理问题(目标失真、梯度放大、赢家通吃饥饿、集合质量崩溃或类别-专家耦合)。我们完全跳出增强动作家族,提出一种解耦代理:一个softmax分类器头以及每个专家独立的sigmoid头,镜像了问题的两个自然对象。我们证明每个样本的更新是坐标式的,且类别-专家Hessian块恒为零,并证明了具有校准常数$\max\{2\sqrt{2},\sqrt{2J/λ}\}$的过量风险界——据我们所知,这是第一个在多专家L2D中当每个专家权重固定时常数不随专家池增长的保证。在受控合成研究以及CIFAR-10、CIFAR-10H和Covertype上,它是我们比较中唯一在专家池增长时保持稳定、保留稀有专家并在每个真实数据基准上优于独立分类器的方法。

英文摘要:

A learning-to-defer (L2D) system decides, for each input, whether to predict on its own or to hand it to one of several available experts. The very well established recipe trains classifier and router jointly by treating the $K$ classes and $J$ experts as competing actions in one shared $(K{+}J)$-action geometry. Subsequent work has proposed a series of incremental fixes within this geometry; we show that each still suffers, to varying severity, from an optimization-level pathology (target distortion, gradient amplification, winner-take-all starvation, set-mass collapse, or class-expert coupling) even under statistical consistency. We step outside the augmented-action family entirely and propose a decoupled surrogate: a softmax classifier head and an independent sigmoid head per expert, mirroring the two natural objects of the problem. We show that per-sample updates are then coordinatewise and the class-expert Hessian block is identically zero, and prove an excess-risk bound with calibration constant $\max\{2\sqrt{2},\sqrt{2J/λ}\}$ -- to our knowledge the first multi-expert L2D guarantee whose constant does not grow with the expert pool when the per-expert weight is held fixed. On controlled synthetic studies and on CIFAR-10, CIFAR-10H, and Covertype, it is the only method in our comparison that remains stable as the expert pool grows, preserves rare specialists, and improves over a standalone classifier on every real-data benchmark.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑