arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2106.02220cs.LGcond-mat.stat-mech
收藏

随机线性学习中的涨落耗散型定理

Fluctuation-dissipation Type Theorem in Stochastic Linear Learning

Seoul National University(首尔国立大学);Cambridge University(剑桥大学);Harvard University(哈佛大学);Sungkyunkwan University(成均馆大学)

Manhyung Han, Jeonghyeok Park, Taewoong Lee, Jung Hoon Han

摘要:

波动耗散定理(FDT)是一个简单而强大的一阶微分方程的结果,该方程控制同时受到耗散力和随机力的系统的动力学。线性学习动态,其中输入向量映射到输出向量的线性矩阵的元素是学习的主题,具有随机版本密切模仿朗之万动态时,一个完整的批量梯度下降方案被替换为随机梯度下降。我们推导了随机线性学习动力学的广义FDT,并在著名的机器学习数据集,如MNIST,CIFAR-10和EMNIST中验证了其有效性。

英文摘要:

The fluctuation-dissipation theorem (FDT) is a simple yet powerful consequence of the first-order differential equation governing the dynamics of systems subject simultaneously to dissipative and stochastic forces. The linear learning dynamics, in which the input vector maps to the output vector by a linear matrix whose elements are the subject of learning, has a stochastic version closely mimicking the Langevin dynamics when a full-batch gradient descent scheme is replaced by that of stochastic gradient descent. We derive a generalized FDT for the stochastic linear learning dynamics and verify its validity among the well-known machine learning data sets such as MNIST, CIFAR-10 and EMNIST.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑