随机线性学习中的涨落耗散型定理
Fluctuation-dissipation Type Theorem in Stochastic Linear Learning
摘要:
波动耗散定理(FDT)是一个简单而强大的一阶微分方程的结果,该方程控制同时受到耗散力和随机力的系统的动力学。线性学习动态,其中输入向量映射到输出向量的线性矩阵的元素是学习的主题,具有随机版本密切模仿朗之万动态时,一个完整的批量梯度下降方案被替换为随机梯度下降。我们推导了随机线性学习动力学的广义FDT,并在著名的机器学习数据集,如MNIST,CIFAR-10和EMNIST中验证了其有效性。
英文摘要:
The fluctuation-dissipation theorem (FDT) is a simple yet powerful consequence of the first-order differential equation governing the dynamics of systems subject simultaneously to dissipative and stochastic forces. The linear learning dynamics, in which the input vector maps to the output vector by a linear matrix whose elements are the subject of learning, has a stochastic version closely mimicking the Langevin dynamics when a full-batch gradient descent scheme is replaced by that of stochastic gradient descent. We derive a generalized FDT for the stochastic linear learning dynamics and verify its validity among the well-known machine learning data sets such as MNIST, CIFAR-10 and EMNIST.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。