SAE：顺序锚定集成
SAE: Sequential Anchored Ensembles
摘要：
由于神经网络参数空间的高维性，计算其贝叶斯后验是一项具有挑战性的任务。锚定集成通过在锚定损失上训练神经网络的集成来近似后验，所述锚定损失被设计用于最优值以遵循贝叶斯后验。然而，训练集合随着其成员数量的增长而变得计算上昂贵，因为对每个成员重复完整的训练过程。在本说明中，我们提出了顺序锚定集成（SAE），一个轻量级的替代锚定集成。不是从头开始训练集合中的每个成员，而是按顺序对具有高自相关性采样的损失进行训练，从而实现神经网络的快速收敛和贝叶斯后验的有效逼近。对于给定的计算预算，SAE在一些基准测试中的表现优于锚定集合，同时在其他基准测试中表现出相当的性能，并在NeurIPS 2021贝叶斯深度学习竞赛中的近似推理的轻型和扩展赛道中获得第二名和第三名。
英文摘要：
Computing the Bayesian posterior of a neural network is a challenging task due to the high-dimensionality of the parameter space. Anchored ensembles approximate the posterior by training an ensemble of neural networks on anchored losses designed for the optima to follow the Bayesian posterior. Training an ensemble, however, becomes computationally expensive as its number of members grows since the full training procedure is repeated for each member. In this note, we present Sequential Anchored Ensembles (SAE), a lightweight alternative to anchored ensembles. Instead of training each member of the ensemble from scratch, the members are trained sequentially on losses sampled with high auto-correlation, hence enabling fast convergence of the neural networks and efficient approximation of the Bayesian posterior. SAE outperform anchored ensembles, for a given computational budget, on some benchmarks while showing comparable performance on the others and achieved 2nd and 3rd place in the light and extended tracks of the NeurIPS 2021 Approximate Inference in Bayesian Deep Learning competition.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
