arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.10360cs.LGcs.CCcs.DScs.LOstat.ML
收藏

直推学习是紧致的

Transductive Learning Is Compact

USC(南加州大学)

Julian Asilis, Siddartha Devic, Shaddin Dughmi, Vatsal Sharan, Shang-Hua Teng

AI总结:

该研究证明直推学习中假设类可学习等价于其有限投影可学习的紧致性对多类损失成立,发现不合适度量损失下可实现学习的紧致性失效并给出2倍差异界,还将结果迁移到PAC模型。

摘要:

我们证明了一个紧致性结果,该结果在带有一类通用损失函数的监督学习中广泛成立:当且仅当假设类$H$的所有有限投影都能以样本复杂度$m$学习时,假设类$H$才能以直推样本复杂度$m$学习。我们证明,这种精确形式的紧致性在可实现学习和不可知学习中均成立,适用于任意合适的度量损失函数(例如$\boldsymbol{\rm{R}}^d$上的任意范数)以及紧致空间上的任意连续损失(例如交叉熵、平方损失)。对于带有不合适度量损失的可实现学习,我们表明样本复杂度的精确紧致性可能不成立,并给出了此类样本复杂度差异程度的匹配上下界,差异因子为2。我们推测在不可知情形下可能存在更大的差距。此外,利用PAC模型与直推模型中样本复杂度的等价性(在可实现情形下,差异仅为低阶因子),我们可以将结果直接迁移到PAC模型,揭示了一种几乎精确的紧致性形式在PAC学习中广泛存在。

英文摘要:

We demonstrate a compactness result holding broadly across supervised learning with a general class of loss functions: Any hypothesis class $H$ is learnable with transductive sample complexity $m$ precisely when all of its finite projections are learnable with sample complexity $m$. We prove that this exact form of compactness holds for realizable and agnostic learning with respect to any proper metric loss function (e.g., any norm on $\mathbb{R}^d$) and any continuous loss on a compact space (e.g., cross-entropy, squared loss). For realizable learning with improper metric losses, we show that exact compactness of sample complexity can fail, and provide matching upper and lower bounds of a factor of 2 on the extent to which such sample complexities can differ. We conjecture that larger gaps are possible for the agnostic case. Furthermore, invoking the equivalence between sample complexities in the PAC and transductive models (up to lower order factors, in the realizable case) permits us to directly port our results to the PAC model, revealing an almost-exact form of compactness holding broadly in PAC learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑