AI 中文总结
研究在非共轭后验上的大规模学习问题，基于常数步长随机梯度上升开发新方法，无需变分后验期望闭式表达式，采用自适应步长，应用于相关数据集时与ResNet特征兼容，聚类指标表现良好。
AI 中文摘要
大规模贝叶斯非参数（BNP）学习者如随机变分推理（SVI）能以较低成本处理大类数和大训练规模的数据集。SVI依赖共轭变分后验假设来近似真实后验。更具挑战性的是在非共轭后验上进行大规模学习，近期相关工作多与蒙特卡罗方法有关且多在非BNP相关任务及简单模型上展示。为克服SVI问题，我们基于常数步长随机梯度上升开发新方法，允许在非共轭后验上大规模学习，无需变分后验期望的闭式表达式，仅要求其可微。受SVI和Adam启发，采用自适应步长显著提升学习效果。应用于MIT67和SUN397等数据集时与ResNet特征兼容，且在聚类指标上能与或优于现有方法。
英文摘要
Large scale Bayesian nonparametrics (BNP) learner such as Stochastic Variational Inference (SVI) can handle datasets with large class number and large training size at fractional cost. Like its predecessor, SVI rely on the assumption of conjugate variational posterior to approximate the true posterior. A more challenging problem is to consider large scale learning on non-conjugate posterior. Recent works in this direction are mostly associated with using Monte Carlo methods for approximating the learner. However, these works are usually demonstrated on non-BNP related task and less complex models such as logistic regression, due to higher computational complexity. In order to overcome the issue faced by SVI, we develop a novel approach based on the recently proposed constant stepsize stochastic gradient ascent to allow large scale learning on non-conjugate posterior. Unlike SVI, our new learner does not require closed- form expression for the variational posterior expectatations. Our only requirement is that the variational posterior is differentiable. In order to ensure convergence in stochastic settings, SVI rely on decaying step-sizes to slow its learning. Inspired by SVI and Adam, we propose the novel use of adaptive stepsizes in our method to significantly improve its learning. We show that our proposed methods is compatible with ResNet features when applied to large class number datasets such as MIT67 and SUN397. Finally, we compare our proposed learner with several recent works such as deep clustering algorithms and showed we were able to produce on-par or outperform the state-of-the-art methods in terms of clustering measures.
