arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.15649cs.LG
收藏

连续时间分段线性递归神经网络

Continuous-Time Piecewise-Linear Recurrent Neural Networks

Department of Theoretical Neuroscience, Central Institute of Mental Health, Mannheim, Germany(理论神经科学系,心理健康中央研究所,曼海姆,德国);Faculty of Physics and Astronomy, Heidelberg University, Heidelberg, Germany(物理和天文学系,海德堡大学,海德堡,德国);Interdisciplinary Center for Scientific Computing, Heidelberg University(科学计算跨学科中心,海德堡大学);Hector Institute for Artificial Intelligence in Psychiatry, Central Institute of Mental Health, Mannheim, Germany(精神病学人工智能研究所,心理健康中央研究所,曼海姆,德国);Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany(数学和计算机科学系,海德堡大学,海德堡,德国)

Alena Brändle, Lukas Eisenmann, Florian Götz, Daniel Durstewitz

AI总结:

本文提出连续时间分段线性递归神经网络(cPLRNNs),通过高效利用其分段线性结构,解决了动态系统重建中连续时间建模和非规则时间间隔数据处理的问题。

摘要:

在动态系统重建(DSR)中,我们的目标是恢复观察到的时间序列背后的动态系统(DS)。具体来说,我们旨在学习一个生成的替代模型,该模型近似了数据生成的DS,并重现其长期特性('气候统计')。在科学和医学领域,这些模型需要具有可机制性——通过其数学分析,我们希望获得对恢复系统运作的见解。分段线性(PL)、基于ReLU的递归神经网络(PLRNNs)在这方面有很强的记录,代表了最先进的DSR模型,同时通过其分段线性设计允许数学洞察。然而,所有当前的PLRNN变体都是离散时间映射。这与大多数物理和生物过程假设的连续时间性质不符,并使难以适应以不规则时间间隔到达的数据。神经ODEs是其中一个解决方案,但它们无法达到PLRNNs的DSR性能,通常缺乏其可追溯性。在这里,我们为连续时间PLRNNs(cPLRNNs)开发了理论:我们提出了一种新的算法,用于训练和模拟此类模型,通过高效利用其分段线性结构,避免数值积分。我们进一步展示了如何通过半解析方法确定训练模型中的重要拓扑对象,如平衡点或极限环。我们比较了cPLRNNs与它们的离散时间近亲以及神经ODEs在DSR基准上的表现,包括具有断点的系统,这些系统带有严格的阈值。

英文摘要:

In dynamical systems reconstruction (DSR) we aim to recover the dynamical system (DS) underlying observed time series. Specifically, we aim to learn a generative surrogate model which approximates the underlying, data-generating DS, and recreates its long-term properties (`climate statistics'). In scientific and medical areas, in particular, these models need to be mechanistically tractable -- through their mathematical analysis we would like to obtain insight into the recovered system's workings. Piecewise-linear (PL), ReLU-based RNNs (PLRNNs) have a strong track-record in this regard, representing SOTA DSR models while allowing mathematical insight by virtue of their PL design. However, all current PLRNN variants are discrete-time maps. This is in disaccord with the assumed continuous-time nature of most physical and biological processes, and makes it hard to accommodate data arriving at irregular temporal intervals. Neural ODEs are one solution, but they do not reach the DSR performance of PLRNNs and often lack their tractability. Here we develop theory for continuous-time PLRNNs (cPLRNNs): We present a novel algorithm for training and simulating such models, bypassing numerical integration by efficiently exploiting their PL structure. We further demonstrate how important topological objects like equilibria or limit cycles can be determined semi-analytically in trained models. We compare cPLRNNs to both their discrete-time cousins as well as Neural ODEs on DSR benchmarks, including systems with discontinuities which come with hard thresholds.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑