鞅后验神经过程
Martingale Posterior Neural Processes
AI总结:
本文提出 MPNP,用神经网络隐式定义的预测分布及鞅后验替代 NP 的有限维潜变量,以更数据驱动地刻画函数不确定性,并在多任务上超过基线。
摘要:
Neural Process(NP,神经过程)在给定数据流的情况下,估计由神经网络隐式定义的随机过程,而不是预先指定已知先验,例如高斯过程。理想的 NP 会在没有任何归纳偏置的情况下从数据中学习一切,但在实践中,为了便于估计,我们通常会限制随机过程的类别。其中一种限制是使用有限维潜变量来解释从 NP 中抽取函数的不确定性。近期一些研究表明,可以通过更“数据驱动”的不确定性来源(如 bootstrap)来改进这一点。在本研究中,我们采用基于鞅后验的不同方法;鞅后验是近期发展起来的贝叶斯推断替代方案。对于鞅后验,我们不指定先验—似然对,而是指定未来数据的预测分布。在预测分布满足特定条件时,可以证明,所生成未来数据中的不确定性实际上对应于隐式定义的贝叶斯后验的不确定性。基于该结果,我们不假设潜变量具有任何形式,而是为 NP 配备一个由神经网络隐式定义的预测分布,并使用相应的鞅后验作为不确定性来源。由此得到的模型被命名为 Martingale Posterior Neural Process(MPNP,鞅后验神经过程),实验证明其在多种任务上优于基线方法。
英文摘要:
A Neural Process (NP) estimates a stochastic process implicitly defined with neural networks given a stream of data, rather than pre-specifying priors already known, such as Gaussian processes. An ideal NP would learn everything from data without any inductive biases, but in practice, we often restrict the class of stochastic processes for the ease of estimation. One such restriction is the use of a finite-dimensional latent variable accounting for the uncertainty in the functions drawn from NPs. Some recent works show that this can be improved with more "data-driven" source of uncertainty such as bootstrapping. In this work, we take a different approach based on the martingale posterior, a recently developed alternative to Bayesian inference. For the martingale posterior, instead of specifying prior-likelihood pairs, a predictive distribution for future data is specified. Under specific conditions on the predictive distribution, it can be shown that the uncertainty in the generated future data actually corresponds to the uncertainty of the implicitly defined Bayesian posteriors. Based on this result, instead of assuming any form of the latent variables, we equip a NP with a predictive distribution implicitly defined with neural networks and use the corresponding martingale posteriors as the source of uncertainty. The resulting model, which we name as Martingale Posterior Neural Process (MPNP), is demonstrated to outperform baselines on various tasks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。