机器学习中训练动态的高斯比较定理
A Gaussian Comparison Theorem for Training Dynamics in Machine Learning
AI总结:
本文提出了一种非渐近的高斯比较定理,用于分析机器学习中训练动态的演变,并展示了动态均值场表达式的有效性及非渐近场景下的改进方法。
摘要:
我们研究了遵循高斯混合模型数据的训练算法。对于特定家族的此类算法,我们提出了一个非渐近结果,将模型的演变连接到一个替代动态系统,该系统更容易分析。我们的结果的证明基于著名的Gordon比较定理。利用我们的定理,我们严格证明了动态均值场(DMF)表达式在渐近场景中的有效性。此外,我们提出了一种迭代细化方案,以在非渐近场景中获得更准确的表达式。我们将我们的理论专门应用于分析使用通用一阶(全批量)算法训练感知机模型,并证明在非渐近域中除了DMF内核外,还会出现波动参数。
英文摘要:
We study training algorithms with data following a Gaussian mixture model. For a specific family of such algorithms, we present a non-asymptotic result, connecting the evolution of the model to a surrogate dynamical system, which can be easier to analyze. The proof of our result is based on the celebrated Gordon comparison theorem. Using our theorem, we rigorously prove the validity of the dynamic mean-field (DMF) expressions in the asymptotic scenarios. Moreover, we suggest an iterative refinement scheme to obtain more accurate expressions in non-asymptotic scenarios. We specialize our theory to the analysis of training a perceptron model with a generic first-order (full-batch) algorithm and demonstrate that fluctuation parameters in a non-asymptotic domain emerge in addition to the DMF kernels.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。