arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.20402stat.MLcs.LG
收藏

后验频率与未拟合的代价:对记忆下界的修正

Posterior Frequencies and the Cost of Not Fitting: A Correction to a Memorization Lower Bound

Purdue University Fort Wayne(普渡大学韦恩堡分校);Nazarbayev University(纳扎尔巴耶夫大学)

Zhenisbek Assylbekov, Rustem Takhanov

AI总结:

本文针对Feldman记忆下界的条件与平均形式给出三点反例,发现其惩罚项与超额风险比值呈指数增长,进而提出用全样本后验概率替代原系数,刻画了适用于无限制学习器的仅多重性最大系数。

摘要:

Feldman的记忆下界会根据某点在训练样本中的出现频率赋予一个频率系数。我们利用独立公平标签和排列等变学习器,给出了该下界的条件形式与平均形式的精确三点反例。一系列采样先验使得所提出的惩罚项与实际超额风险的比值呈指数增长,排除了通用正常数因子修正的可能性。这些先验随样本量变化,违规样本变得稀少,且两个量均趋于零。保留原模型,我们使用全样本后验概率得到了精确替代方案。在可交换性下,这些概率依赖于多重性和完整频率分布。我们刻画了适用于无限制学习器的仅基于多重性的最大系数。原系数具有Good精确频率公式的后验均值形式,是修正后系数的占用加权平均。

英文摘要:

Feldman's memorization lower bound assigns a frequency coefficient based on how often a point appears in the training sample. We give an exact three-point counterexample to its conditional and averaged forms, using independent fair labels and a permutation-equivariant learner. A sequence of sampling priors makes the ratio of the proposed penalty to actual excess risk grow exponentially, ruling out a universal positive constant-factor repair. The priors vary with sample size, the offending profiles become rare, and both quantities vanish. Retaining the original model, we obtain an exact replacement using full-sample posterior probabilities. Under exchangeability, these depend on multiplicity and the complete frequency profile. We characterize the largest multiplicity-only coefficients valid for unrestricted learners. The original coefficient has the posterior-mean form of Good's exact frequency formula and is an occupancy-weighted average of the corrected coefficients.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑