arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2606.04045cs.LGcs.AI
收藏

监督学习中的贝叶斯充分表示

Bayes-Sufficient Representations in Supervised Learning

Athena Research Center, Kimmeria Campus, Xanthi, Greece(阿塔尼亚研究中心,基米里亚校区,辛提斯,希腊);Democritus University of Thrace, Vas. Sofias Campus, Xanthi, Greece(德摩根大学,瓦斯·索菲亚校区,辛提斯,希腊);International Hellenic University, Serres, Greece(国际希腊大学,塞雷斯,希腊)

Vasileios Sevetlidis

AI总结:

本文定义了监督学习中表示对损失函数的贝叶斯充分性,引入贝叶斯商概念,并证明最小充分表示等价于贝叶斯商,通过实验区分了充分性、最小性和非必要信息保留。

摘要:

表示学习通常被描述为保留输入中与预测相关的信息。本文探讨了在固定监督决策问题中相关性的含义。定义了一个表示对于联合分布和损失是贝叶斯充分的,如果某个预测头可以使用它来实现贝叶斯最优行动规则。这使得目标信息依赖于损失。在几乎必然唯一的贝叶斯行动情况下,相关对象是贝叶斯商,它识别需要相同贝叶斯最优行动的输入。当表示细化这个商时,它是充分的;当它在信息上等价于商时,它是贝叶斯最小的。该框架自然地连接到属性诱导:零一损失需要贝叶斯类,平方损失需要条件均值,布里尔损失需要二元预测中的条件概率,对数损失或严格适当评分规则需要预测分布。受控的有限实验、学习的神经瓶颈实验以及真实数据的iNaturalist分类学细化实验说明了充分性、最小性和保留的非必要信息之间的区别。对于固定的监督问题,分布和损失决定贝叶斯行动,贝叶斯行动决定商,商决定贝叶斯最优预测所需的最小信息。

英文摘要:

Representation learning is often described as preserving the information in an input that is relevant for prediction. This work asks what relevance means for a fixed supervised decision problem. A representation is defined to be Bayes-sufficient for a joint distribution and loss if some prediction head can use it to implement a Bayes-optimal action rule. This makes the target information loss-dependent. In the almost-surely unique Bayes-action case, the relevant object is a Bayes quotient, which identifies inputs that require the same Bayes-optimal action. A representation is sufficient when it refines this quotient, and Bayes-minimal when it is informationally equivalent to it. The framework connects naturally to property elicitation: zero-one loss requires the Bayes class, squared loss the conditional mean, Brier loss the conditional probability in binary prediction, and log loss or strictly proper scoring rules the predictive distribution. Controlled finite experiments, learned neural bottleneck experiments, and a real-data iNaturalist taxonomic refinement experiment illustrate the distinction between sufficiency, minimality, and retained non-required information. For a fixed supervised problem, the distribution and the loss determine the Bayes action, the Bayes action determines the quotient, and the quotient determines the minimal information required for Bayes-optimal prediction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑