发表机构
Osaka Metropolitan University; International Research Center for Neurointelligence (IRCN), The University of Tokyo; Nagoya Institute of Technology（大阪公立大学; 东京大学国际神经智能研究中心; 名古屋工业大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出不动点引导的层次化约简方法，通过先训练冗余AL-RNN再逐步线性化约简，实现最小动力学表示，并在蔡氏系统上将成功率从20%提升至约71%。
AI 中文摘要
从时间序列理解非线性动力学系统，不仅需要重现其轨迹，还需要识别出保留其本质动力学结构的简单表示。几乎线性递归神经网络（AL-RNNs）是分段线性RNN，其中仅一部分单元使用ReLU非线性，从而通过ReLU单元的数量显式控制非线性容量。它们的激活模式定义了线性区域，以符号表示，其观测到的转移形成符号转移图。然而，直接使用少量ReLU单元训练AL-RNN以实现最小动力学表示可能不可靠。我们提出一个问题：是否可以先训练具有更多ReLU单元的AL-RNN，然后系统地将其约简为最小动力学表示。我们引入了一种不动点引导的层次化约简过程，逐步线性化选定的ReLU单元，合并相邻的线性区域和图节点，同时保留包含不动点（FPs）的不同符号。由此产生的约简树定义了逐步更简单候选的层次结构。每个约简候选从父参数初始化，并在父动力学指导下重新训练。我们还证明了重现$Q$个不同不动点至少需要$Q$个包含FP的符号，当达到此界限时，提供了符号级最小性的证书。在3-涡卷蔡氏系统上，使用理论最少三个ReLU单元直接训练，仅在20%的种子中实现高保真最小实现，而我们的学习-约简-再训练策略在相同的最终非线性容量下将种子宏成功率提高到约71%。这些结果表明，冗余的非线性容量可以作为发现和实现最小动力学表示的支架。
英文摘要
Understanding a nonlinear dynamical system from time series requires not only reproducing its trajectories, but also identifying a simple representation that preserves its essential dynamical structure. Almost-linear recurrent neural networks (AL-RNNs) are piecewise-linear RNNs in which only a subset of units use ReLU nonlinearities, so that nonlinear capacity is explicitly controlled by the number of ReLU units. Their activation patterns define linear regions, represented as symbols, whose observed transitions form a symbolic transition graph. However, directly training AL-RNNs with few ReLU units to realize minimal dynamical representations can be unreliable. We ask whether an AL-RNN with more ReLU units can instead be trained first and systematically reduced to a minimal dynamical representation. We introduce a fixed-point-guided hierarchical reduction procedure that progressively linearizes selected ReLU units, merging neighboring linear regions and graph nodes while preserving distinct symbols containing fixed points (FPs). The resulting reduction tree defines a hierarchy of progressively simpler candidates. Each reduced candidate is initialized from the parent parameters and retrained under guidance from the parent dynamics. We also prove that reproducing $Q$ distinct fixed points requires at least $Q$ FP-containing symbols, providing a certificate of symbol-level minimality when this bound is attained. On the 3-scroll Chua system, direct training with the theoretical minimum of three ReLU units achieves high-fidelity minimal realizations in only 20% of seeds, whereas our learn-reduce-retrain strategy increases the seed-macro success rate to approximately 71% at the same final nonlinear capacity. These results show that redundant nonlinear capacity can serve as a scaffold for discovering and realizing minimal dynamical representations.
Comments27 pages, 6 figures
