arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.17205stat.MLcs.DScs.LG
收藏

近优算法用于全能预测

Near-Optimal Algorithms for Omniprediction

Princewill Okoroafor, Robert Kleinberg, Michael P. Kim

AI总结:

本文提出近最优算法用于全能预测,通过在线和离线方法实现对多种损失函数类的高效预测。

摘要:

全能预测器是简单的预测函数,它们在假设类H中针对每个损失函数类L中的损失最小化预测进行编码。在本工作中,我们为全能预测提供了近最优的学习算法,包括在线和离线设置。首先,我们给出一个oracle高效的在线学习算法,能够以~O(√(T log|H|))的后悔率实现(L,H)-全能预测,对于任何Lipschitz损失函数类L⊆L_Lip。令人惊讶的是,这个后悔界与单个损失函数最小化的最优后悔界(除以√log(T)因子外)相匹配。鉴于这个在线算法,我们开发了一个在线到离线的转换,能够在多种度量上实现近最优的复杂度。特别是,对于所有有界变化损失函数类L_BV(包括所有凸函数、所有Lipschitz函数和所有proper损失)以及任何(可能无限的)H,我们获得了一个离线学习算法,利用一个(离线)ERM oracle和m个样本从D中,返回一个高效的(L_BV,H,ε(m))-全能预测器,其中ε(m)以Rademacher复杂度的类H的衍生类的凸组合的固定数量元素的Rademacher复杂度的近线性方式缩放。

英文摘要:

Omnipredictors are simple prediction functions that encode loss-minimizing predictions with respect to a hypothesis class $H$, simultaneously for every loss function within a class of losses $L$. In this work, we give near-optimal learning algorithms for omniprediction, in both the online and offline settings. To begin, we give an oracle-efficient online learning algorithm that acheives $(L,H)$-omniprediction with $\tilde O (\sqrt{T \log |H|})$ regret for any class of Lipschitz loss functions $L \subseteq L_\mathrm{Lip}$. Quite surprisingly, this regret bound matches the optimal regret for \emph{minimization of a single loss function} (up to a $\sqrt{\log(T)}$ factor). Given this online algorithm, we develop an online-to-offline conversion that achieves near-optimal complexity across a number of measures. In particular, for all bounded loss functions within the class of Bounded Variation losses $L_\mathrm{BV}$ (which include all convex, all Lipschitz, and all proper losses) and any (possibly-infinite) $H$, we obtain an offline learning algorithm that, leveraging an (offline) ERM oracle and $m$ samples from $D$, returns an efficient $(L_{\mathrm{BV}},H,ε(m))$-omnipredictor for $\varepsilon(m)$ scaling near-linearly in the Rademacher complexity of a class derived from $H$ by taking convex combinations of a fixed number of elements of $\mathrm{Th} \circ H$.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑