arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2505.23947cs.LGcs.AI
收藏

立场:贝叶斯预测的未来是先验拟合

Position: The Future of Bayesian Prediction Is Prior-Fitted

University of Freiburg(弗莱堡大学);Meta;LMU Munich(慕尼黑大学);Prior Labs;Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心)

Samuel Müller, Arik Reuter, Noah Hollmann, David Rügamer, Frank Hutter

AI总结:

本文指出在预训练算力增长而真实数据稀缺的背景下,先验数据拟合网络(PFNs)通过摊销学习将计算资源高效分配至低数据场景,代表了贝叶斯推理的未来,并探讨了其潜力与局限性。

摘要:

在随机生成的人工数据集上训练神经网络,能够产生捕获由数据集生成分布所定义的先验的贝叶斯模型。先验数据拟合网络是一类旨在利用这一见解的方法。在用于预训练的计算资源快速增长,以及许多应用中新真实世界数据的生成近乎停滞的时代,PFN有望在广泛的应用中发挥更重要的作用。它们能够将预训练计算资源高效分配到低数据场景中。最初应用于小型贝叶斯建模任务,PFN领域已显著扩展,以应对更复杂的领域和更大的数据集。本立场论文认为,PFN和其他摊销推理方法代表了贝叶斯推理的未来,利用摊销学习来解决数据稀缺问题。因此,我们认为它们是一个富有成果的研究领域。在本立场论文中,我们探讨了它们的潜力以及解决其当前局限性的方向。

英文摘要:

Training neural networks on randomly generated artificial datasets yields Bayesian models that capture the prior defined by the dataset-generating distribution. Prior-data Fitted Networks (PFNs) are a class of methods designed to leverage this insight. In an era of rapidly increasing computational resources for pre-training and a near stagnation in the generation of new real-world data in many applications, PFNs are poised to play a more important role across a wide range of applications. They enable the efficient allocation of pre-training compute to low-data scenarios. Originally applied to small Bayesian modeling tasks, the field of PFNs has significantly expanded to address more complex domains and larger datasets. This position paper argues that PFNs and other amortized inference approaches represent the future of Bayesian inference, leveraging amortized learning to tackle data-scarce problems. We thus believe they are a fruitful area of research. In this position paper, we explore their potential and directions to address their current limitations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑