量化特权信息价值：一种PAC-Bayesian方法
Quantifying the Value of Privileged Information Using a PAC-Bayesian Approach
- Leiden University（莱顿大学）
- Honda Research Institute Europe GmbH（本田欧洲研究院有限公司）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文提出一种基于PAC-Bayes的算法无关信息论方法，通过比较有无特权信息时的最紧风险界来量化其潜在价值，并引入训练时指标，无需测试数据即可预测性能提升。
AI中文摘要：
在实践中，各种学习场景仅在训练期间提供对辅助特征的访问。整合此类数据以增强模型性能催生了一种被称为“利用特权信息学习”（LUPI）的范式。虽然这种额外信息旨在改善最终模型，但建立关于特权信息（PI）如何传递有用知识的通用、连贯的理解仍然是一个挑战。Vapnik的原始理论及后续工作在某些情况下提供了性能保证，但这些结果本质上是针对特定算法的，并依赖于特定设置的证明方法。因此，一个更通用的框架来解释PI如何以及何时传递有用知识仍然缺失。为弥补这一空白，我们引入了一种基于PAC-Bayes框架的、与算法无关的信息论方法。我们不是问特定算法是否利用了PI，而是问它能提供多少价值：比较有无PI时最紧的可行风险界，得出其潜力——可提取收益的上限。我们引入了一个直接从经验训练风险量化这种潜力的指标，无需访问测试时数据，并在监督和无监督设置中验证了我们的发现。结果表明，我们的训练时指标与真实测试时性能提升之间存在稳健的对应关系。最终，这项工作朝着对LUPI的信息论理解迈出了必要的一步，并在承诺使用模型之前量化特权特征的潜力。
英文摘要：
In practice, various learning scenarios provide access to auxiliary features exclusively during training. Incorporating such data to enhance model performance gave rise to a paradigm known as Learning Using Privileged Information (LUPI). While this extra information is intended to improve the resulting model, establishing a generalized, cohesive understanding of how privileged information (PI) transfers useful knowledge remains a challenge. Vapnik's original theory and subsequent works offer performance guarantees in certain cases, but these results are inherently per-algorithm and rely on setting-specific proof approaches. Consequently, a more general framework explaining how and when PI transfers useful knowledge is still missing. To bridge this gap, we introduce an algorithm-agnostic, information-theoretic approach based on the PAC-Bayes framework. Rather than asking whether a particular algorithm exploits PI, we ask how much value it could offer: comparing the tightest achievable risk bound with and without PI yields its potential - an upper limit on the extractable gain. We introduce a metric that quantifies this potential directly from empirical training risk, bypassing the need for test-time data access, and validate our findings in both supervised and unsupervised settings. The results demonstrate a robust correspondence between our training-time metric and true test-time performance gains. Ultimately, this work takes a necessary step toward an information-theoretic understanding of LUPI, and quantifying the potential of privileged features before committing to a model.
