arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2101.00245stat.MLcs.CVcs.LGcs.NE
收藏

张量网络的贝叶斯方法

The Bayesian Method of Tensor Networks

University of California, Santa Cruz(加州大学圣克鲁兹分校)

Erdong Guo, David Draper

AI总结:

本文提出基于张量网络的贝叶斯方法,通过引入先验分布和稳定初始化技巧,提升模型的泛化能力和预测性能。

摘要:

贝叶斯学习是一种强大的学习框架,它以一种在推理和预测中逻辑一致的方式将数据的外部信息(背景信息)与内部信息(训练数据)结合起来。通过贝叶斯规则,外部信息(先验分布)和内部信息(训练数据似然)被结合起来,从而得到后验分布和后验预测(边际)分布,分别总结了推理和预测所需的全部信息。在本文中,我们从两个角度研究张量网络的贝叶斯框架。首先,我们将先验分布引入到张量网络的权重中,并通过后验预测(边际)分布预测新观测的标签。由于在归一化常数计算中参数积分的不可计算性,我们通过拉普拉斯近似法近似后验预测分布,并获得张量网络模型后验分布的海森矩阵的外积近似值。其次,为了估计静态模式的参数,我们提出了一种稳定的初始化技巧,通过这种方法,张量网络可以更高效且稳定地收敛到静态路径。我们验证了我们的工作在MNIST、钓鱼网站和乳腺癌数据集上的效果。我们通过可视化模型参数和二维合成数据集中的决策边界来研究贝叶斯张量网络的贝叶斯特性。从应用目的来看,我们的工作可以减少过拟合并提高正常张量网络模型的性能。

英文摘要:

Bayesian learning is a powerful learning framework which combines the external information of the data (background information) with the internal information (training data) in a logically consistent way in inference and prediction. By Bayes rule, the external information (prior distribution) and the internal information (training data likelihood) are combined coherently, and the posterior distribution and the posterior predictive (marginal) distribution obtained by Bayes rule summarize the total information needed in the inference and prediction, respectively. In this paper, we study the Bayesian framework of the Tensor Network from two perspective. First, we introduce the prior distribution to the weights in the Tensor Network and predict the labels of the new observations by the posterior predictive (marginal) distribution. Since the intractability of the parameter integral in the normalization constant computation, we approximate the posterior predictive distribution by Laplace approximation and obtain the out-product approximation of the hessian matrix of the posterior distribution of the Tensor Network model. Second, to estimate the parameters of the stationary mode, we propose a stable initialization trick to accelerate the inference process by which the Tensor Network can converge to the stationary path more efficiently and stably with gradient descent method. We verify our work on the MNIST, Phishing Website and Breast Cancer data set. We study the Bayesian properties of the Bayesian Tensor Network by visualizing the parameters of the model and the decision boundaries in the two dimensional synthetic data set. For a application purpose, our work can reduce the overfitting and improve the performance of normal Tensor Network model.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑