arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2206.03992stat.MLcs.LG
收藏

神经扩散过程

Neural Diffusion Processes

University of Cambridge(剑桥大学);Secondmind(Secondmind公司);Google DeepMind(谷歌DeepMind)

Vincent Dutordoir, Alan Saul, Zoubin Ghahramani, Fergus Simpson

AI总结:

提出 Neural Diffusion Processes(NDPs),利用去噪扩散和定制注意力对函数分布进行元学习,以逼近贝叶斯后验并支持回归、优化等下游任务。

摘要:

用于对函数分布进行元学习的神经网络方法具有一些理想特性,例如更高的灵活性和更低的推理复杂度。基于去噪扩散模型在生成建模中的成功,我们提出了 Neural Diffusion Processes(NDPs,神经扩散过程),这是一种通过有限边际来学习从丰富的函数分布中采样的新方法。通过引入定制注意力模块,我们能够将随机过程的特性(如可交换性)直接融入 NDP 的架构中。我们通过实验表明,NDPs 可以捕获接近真实贝叶斯后验的函数分布,证明它们能够成功模拟高斯过程的行为,并超越神经过程的性能。NDPs 支持多种下游任务,包括回归、隐式超参数边际化、非高斯后验预测和全局优化。

英文摘要:

Neural network approaches for meta-learning distributions over functions have desirable properties such as increased flexibility and a reduced complexity of inference. Building on the successes of denoising diffusion models for generative modelling, we propose Neural Diffusion Processes (NDPs), a novel approach that learns to sample from a rich distribution over functions through its finite marginals. By introducing a custom attention block we are able to incorporate properties of stochastic processes, such as exchangeability, directly into the NDP's architecture. We empirically show that NDPs can capture functional distributions close to the true Bayesian posterior, demonstrating that they can successfully emulate the behaviour of Gaussian processes and surpass the performance of neural processes. NDPs enable a variety of downstream tasks, including regression, implicit hyperparameter marginalisation, non-Gaussian posterior prediction and global optimisation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑