arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.08733cs.LG
收藏

专家不作弊:通过预测配对学习未知知识

Experts Don't Cheat: Learning What You Don't Know By Predicting Pairs

Google DeepMind(谷歌DeepMind);University of Toronto(多伦多大学)

Daniel D. Johnson, Daniel Tarlow, David Duvenaud, Chris J. Maddison

AI总结:

提出训练模型预测配对响应并允许“作弊”的策略,通过衡量作弊程度实现二阶校准,从而准确估计认知不确定性,在图像分类、语言建模和导航任务中优于现有方法。

摘要:

识别模型 ${\widehat{p}}_θ(Y|X)$ 对其训练所依据的随机真实世界过程 $p(Y|X)$ 的了解程度,对于确保其避免产生不正确或“幻觉”答案或采取不安全行动至关重要。但对于生成模型而言这很困难,因为概率预测无法区分逐响应噪声(偶然不确定性)和对过程知识的缺乏(认知不确定性),且现有的认知不确定性量化技术在模型欠拟合时往往过于自信。我们提出了一种通用策略,用于教会模型既近似 $p(Y|X)$,又估计 ${\widehat{p}}_θ(Y|X)$ 与 $p(Y|X)$ 之间的剩余差距:训练它预测从真实条件分布中抽取的独立响应配对,允许其通过观察一个响应来预测另一个响应而“作弊”,然后衡量其作弊程度。值得注意的是,我们证明了擅长作弊(即只要作弊能改进预测就作弊)等价于二阶校准,这是普通校准的一个原则性扩展,使我们能够为 $p(Y|X)$ 构造可证明正确的频率学置信区间,并以高概率检测错误响应。我们通过实验证明,我们的方法在模糊图像分类、(合成)语言建模和部分可观测导航任务中能准确估计模型未知程度,优于现有技术。

英文摘要:

Identifying how much a model ${\widehat{p}}_θ(Y|X)$ knows about the stochastic real-world process $p(Y|X)$ it was trained on is important to ensure it avoids producing incorrect or "hallucinated" answers or taking unsafe actions. But this is difficult for generative models because probabilistic predictions do not distinguish between per-response noise (aleatoric uncertainty) and lack of knowledge about the process (epistemic uncertainty), and existing epistemic uncertainty quantification techniques tend to be overconfident when the model underfits. We propose a general strategy for teaching a model to both approximate $p(Y|X)$ and also estimate the remaining gaps between ${\widehat{p}}_θ(Y|X)$ and $p(Y|X)$: train it to predict pairs of independent responses drawn from the true conditional distribution, allow it to "cheat" by observing one response while predicting the other, then measure how much it cheats. Remarkably, we prove that being good at cheating (i.e. cheating whenever it improves your prediction) is equivalent to being second-order calibrated, a principled extension of ordinary calibration that allows us to construct provably-correct frequentist confidence intervals for $p(Y|X)$ and detect incorrect responses with high probability. We demonstrate empirically that our approach accurately estimates how much models don't know across ambiguous image classification, (synthetic) language modeling, and partially-observable navigation tasks, outperforming existing techniques.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑