未知顺序上阈值的 ERM-预言机复杂度中的指数级确定性与随机性差距

中文摘要 AI 辅助

Attias、Hanneke 和 Ramaswami（NeurIPS 2025）提出一个问题：当类别仅能通过预言机访问时，随机化是否能在在线学习中显著减少所需的预言机调用次数。我们研究了他们特别指出的实例：在 T 个实例的未知全序上进行转导在线学习，使用一致性类型的 ERM 预言机，该预言机返回与查询标记集一致的全部概念（或报告不可实现性）。我们的主要结果是针对固定自然预言机的分离。当预言机采用最小前缀规则（或最大前缀规则）时，每个确定性学习器在某些实例上犯 M 个错误并进行 Q 次调用，满足 M+Q ≥ T−ε（ε∈{0,1}，取决于空前缀是否是一个概念），且该常数是精确的；因此，O(log T) 个错误需要 T−ε−O(log T) 次调用，而该论文中的随机学习器在同一规则下实现了 O(log T) 的期望调用和错误。随机顺序是最优的：在最小前缀规则下的显式困难分布上，每个学习器的期望错误至少为 ((T+1−ε)·128^{−E[Q]}−1)/2，因此对于多对数错误，Ω(log T) 的期望调用是必要的。这种分离由预言机的选择规则决定，而非仅由类别本身决定：对于合法的可行中位数 ERM 规则，确定性学习器可以实现 O(log T) 的调用和错误，而全局中位数规则再次迫使线性总成本。当预言机的答案由对抗性选择并冻结为无记忆预言机时，同样的线性界限成立。我们补充了固定查询预算的部分权衡结果（中间区域是开放的），并进行了接口对比：仅使用弱一致性预言机（返回可实现性位）时，确定性和随机学习器都需要 Θ(T) 次调用。

英文摘要

Attias, Hanneke and Ramaswami (NeurIPS 2025) asked whether randomization provably reduces the oracle calls needed for online learning when the class is accessible only through an oracle. We study the instance they singled out: transductive online learning of thresholds on an unknown total order of T instances, with a consistency-type ERM oracle that returns a full concept consistent with a queried labeled set (or reports non-realizability). Our main result is a separation for a fixed natural oracle. When the oracle is the minimal-prefix rule (or the maximal-prefix rule), every deterministic learner makes M mistakes and Q calls with $M+Q\ge T-\varepsilon$ on some instance ($\varepsilon\in\{0,1\}$, according to whether the empty prefix is a concept), and the constant is exact; hence $O(\log T)$ mistakes cost $T-\varepsilon-O(\log T)$ calls, whereas that paper's randomized learner achieves $O(\log T)$ expected calls and mistakes under the same rule. The randomized order is optimal: on an explicit hard distribution under the minimal-prefix rule, every learner has expected mistakes at least $((T+1-\varepsilon)\,128^{-\mathbb{E}[Q]}-1)/2$, so $Ω(\log T)$ expected calls are necessary for polylogarithmic mistakes. The separation is governed by the oracle's selection rule, not by the class alone: for a legal feasible-median ERM rule a deterministic learner achieves $O(\log T)$ calls and mistakes, while a global-median rule again forces linear total cost. The same linear bound holds when the oracle's answers are chosen adversarially and then frozen into a memoryless oracle. We add partial tradeoff results for fixed query budgets (the middle regime is open) and an interface contrast: with only a weak consistency oracle, returning a realizability bit, both deterministic and randomized learners need $Θ(T)$ calls.

发表机构

• University of New South Wales（新南威尔士大学）

机构由 AI 辅助整理，请以论文原文为准。