CHLU:作为深度学习的辛原初的因果哈密顿学习单元
CHLU: The Causal Hamiltonian Learning Unit as a Symplectic Primitive for Deep Learning
AI总结:
本文提出CHLU,一种基于物理的计算学习原初,通过 enforcing 相对论哈密顿结构和利用辛积分,严格保持相空间体积,以解决记忆稳定性权衡问题,并在MNIST数据集上展示其生成能力。
摘要:
当前处理时间动态的深度学习原初面临根本二元对立:它们要么是离散且不稳定的(LSTMs)[1],导致梯度爆炸或消失;要么是连续且耗散的(神经ODEs)[2],通过时间破坏信息以确保稳定性。我们提出因果哈密顿学习单元(pronounced: clue),一种新颖的基于物理的计算学习原初。通过 enforcing 相对论哈密顿结构并利用辛积分,CHLU严格保持相空间体积,作为解决记忆稳定性权衡的尝试。我们展示CHLU设计用于无限时间稳定性,以及可控噪声过滤。然后,我们使用MNIST数据集作为证明原理,演示CHLU的生成能力。
英文摘要:
Current deep learning primitives dealing with temporal dynamics suffer from a fundamental dichotomy: they are either discrete and unstable (LSTMs) \citep{pascanu_difficulty_2013}, leading to exploding or vanishing gradients; or they are continuous and dissipative (Neural ODEs) \citep{dupont_augmented_2019}, which destroy information over time to ensure stability. We propose the \textbf{Causal Hamiltonian Learning Unit} (pronounced: \textit{clue}), a novel Physics-grounded computational learning primitive. By enforcing a Relativistic Hamiltonian structure and utilizing symplectic integration, a CHLU strictly conserves phase-space volume, as an attempt to solve the memory-stability trade-off. We show that the CHLU is designed for infinite-horizon stability, as well as controllable noise filtering. We then demonstrate a CHLU's generative ability using the MNIST dataset as a proof-of-principle.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。