arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.19802cs.LG
收藏

学习参数化概率模型的随机热力学

Stochastic Thermodynamics of Learning Parametric Probabilistic Models

City University of New York(纽约城市大学)

Shervin Sadat Parsi

AI总结:

该研究将一类机器学习问题表述为参数化概率模型的时间演化热力学过程,引入记忆信息与学习信息两种度量追踪学习信息流,证明学习信息累积与熵产生相关、参数以记忆信息形式捕获所学内容,为评估概率模型学习的信息论内涵提供了热力学视角的工具。

摘要:

我们将一类机器学习问题表述为参数化概率模型(Parametric Probabilistic Models,简称PPMs)的时间演化过程,该过程本质上构成了一个热力学过程。我们的核心动机是利用信息热力学的丰富工具集,来评估学习一个概率模型的信息论内涵。我们首先引入两种信息论度量:记忆信息(Memorized-information,简称M-info)和学习信息(Learned-information,简称L-info),二者可追踪PPMs学习过程中的信息流。随后我们证明,学习过程中L-info的累积与熵产生相关,而参数在该过程中充当热库,以M-info的形式捕获学习到的信息。

英文摘要:

We have formulated a family of machine learning problems as the time evolution of Parametric Probabilistic Models (PPMs), inherently rendering a thermodynamic process. Our primary motivation is to leverage the rich toolbox of thermodynamics of information to assess the information-theoretic content of learning a probabilistic model. We first introduce two information-theoretic metrics: Memorized-information (M-info) and Learned-information (L-info), which trace the flow of information during the learning process of PPMs. Then, we demonstrate that the accumulation of L-info during the learning process is associated with entropy production, and parameters serve as a heat reservoir in this process, capturing learned information in the form of M-info.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑