arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2206.11533math.OCcs.LGcs.NAmath.NA
收藏

随机朗之万微分包含及其在机器学习中的应用

Stochastic Langevin Differential Inclusions with Applications to Machine Learning

Fabio V. Difonzo, Vyacheslav Kungurtsev, Jakub Marecek

AI总结:

针对机器学习中势函数非光滑导致漂移项不满足利普希茨连续的问题,研究朗之万型随机微分包含的性质,证明了解的强存在性与正则自由能泛函的渐近最小化,为非光滑场景下的相关算法提供理论基础。

摘要:

朗之万扩散形式的随机微分方程因其在贝叶斯采样算法和机器学习优化中的基础作用而受到广泛关注。在后者中,它们作为训练过参数化模型时随机梯度流的概念模型。然而,现有文献通常假设势函数是光滑的,其梯度即为漂移项。但在许多问题中,势函数并非连续可微,因此漂移项并非处处满足利普希茨连续性,回归问题中的鲁棒损失和修正线性单元(Rectified Linear Units)就是典型例子。本文在适用于机器学习场景的假设下,给出了朗之万型随机微分包含的流性质与渐近性质的若干基础结果。具体而言,我们证明了解的强存在性,以及正则自由能泛函的渐近最小化特性。

英文摘要:

Stochastic differential equations of Langevin-diffusion form have received significant attention, thanks to their foundational role in both Bayesian sampling algorithms and optimization in machine learning. In the latter, they serve as a conceptual model of the stochastic gradient flow in training over-parameterized models. However, the literature typically assumes smoothness of the potential, whose gradient is the drift term. Nevertheless, there are many problems for which the potential function is not continuously differentiable, and hence the drift is not Lipschitz continuous everywhere. This is exemplified by robust losses and Rectified Linear Units in regression problems. In this paper, we show some foundational results regarding the flow and asymptotic properties of Langevin-type Stochastic Differential Inclusions under assumptions appropriate to the machine-learning settings. In particular, we show strong existence of the solution, as well as an asymptotic minimization of the canonical free-energy functional.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑