arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.18577cs.LGcs.AI
收藏

当更多目标使识别变得更容易:偏好与反馈的作用

When More Objectives Make Identification Easier: The Role of Preferences and Feedback

City University of Hong Kong(香港城市大学);Zhejiang University(浙江大学)

Bo Xue, Ji Cheng, Yuanyu Wan, Shuang Qiu

AI总结:

本文研究多目标最佳臂识别问题,通过理论分析揭示多目标可减少样本量的三种现象,并提出Order-Aware Track-and-Stop算法,其性能渐近匹配理论下界。

摘要:

最佳臂识别(BAI)旨在使用尽可能少的样本识别最优臂。虽然大多数现有工作聚焦于单一目标,但许多现实应用涉及多个标准,这促使人们研究多目标老虎机中的BAI。直观来看,多目标识别似乎更困难,因为学习器必须在更高维的奖励空间中识别最优臂;然而,额外的目标也提供了更丰富的反馈,可减少所需的探索。在本文中,我们明确了多目标何时以及为何能使BAI变得更容易。对于高斯老虎机,我们建立了固定置信度识别的特征时间下界,该下界在所有偏好序下均成立。通过对帕累托、线性和词典序偏好的显式比较,我们揭示了三种现象:(i)识别更大的最优集可能比识别唯一最优解需要更少的样本;(ii)向量反馈相对于坐标反馈的优势根本上取决于偏好序;(iii)多目标识别所需的样本量甚至可能比对应的最简单单目标问题更少。最后,我们提出了Order-Aware Track-and-Stop(OATS)算法,该算法在任何规定的置信水平下均正确,且渐近匹配该下界。

英文摘要:

Best-arm identification (BAI) aims to identify the optimal arm using as few samples as possible. While most existing work focuses on a single objective, many real-world applications involve multiple criteria, which motivates the study of BAI in multi-objective bandits. Intuitively, multi-objective identification seems harder, since the learner must identify optimal arms in a higher-dimensional reward space; however, the additional objectives also provide richer feedback that can reduce the required exploration. In this paper, we characterize when and why multiple objectives can make BAI easier. For Gaussian bandits, we establish a characteristic-time lower bound for fixed-confidence identification that holds uniformly across preference orders. Through explicit comparisons under Pareto, linear, and lexicographic preferences, we reveal three phenomena: (i) identifying a larger optimal set can require fewer samples than identifying a unique optimum; (ii) the advantage of vector feedback over coordinate feedback depends fundamentally on the preference order; and (iii) multi-objective identification can require fewer samples than even the easiest corresponding single-objective problem. Finally, we propose Order-Aware Track-and-Stop (OATS), an algorithm that is correct at any prescribed confidence level and asymptotically matches the lower bound.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑