arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.04934cs.LGcs.AImath.DSnlin.CD
收藏

用于动力系统重构的最优循环网络拓扑

Optimal Recurrent Network Topologies for Dynamical Systems Reconstruction

Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University(海德堡大学曼海姆医学院中央精神卫生研究所);Heidelberg University(海德堡大学);Interdisciplinary Center for Scientific Computing, Heidelberg University(海德堡大学跨学科科学计算中心)

Christoph Jürgen Hemmer, Manuel Brenner, Florian Hess, Daniel Durstewitz

AI总结:

本文提出几何剪枝方法,通过移除对吸引子几何结构贡献低的权重来降低动力系统重构中的参数负载,并发现由此产生的特定拓扑对性能最为关键,进而给出自动生成该最优拓扑的算法。

摘要:

在动力系统重构(DSR)中,我们试图从时间序列测量中推断出底层动力过程的生成模型。这是任何科学学科中的首要目标,我们尤其关注参数负载较低的简约模型。这里的一种常见策略是参数剪枝,即移除所有权重较小的参数。然而,我们发现这种策略对 DSR 并不适用,因为即使是低幅值的参数也可能对系统动力学产生相当大的贡献。另一方面,众所周知,许多产生复杂动力学的自然系统(如大脑或生态网络)具有稀疏拓扑,连接相对较少。受此启发,我们表明几何剪枝——与基于幅值的剪枝不同,它移除对吸引子几何结构贡献较低的权重——确实能够在不显著损害 DSR 质量的情况下大幅降低参数负载。我们进一步发现,几何剪枝得到的网络具有特定类型的拓扑,而这种拓扑(而非权重幅值)对性能最为关键。我们提供了一种自动生成此类拓扑的算法,该拓扑可用作 RNN 对动力系统进行生成建模的先验,并将其与小世界网络或无标度网络等其他经过充分研究的拓扑进行比较。

英文摘要:

In dynamical systems reconstruction (DSR) we seek to infer from time series measurements a generative model of the underlying dynamical process. This is a prime objective in any scientific discipline, where we are particularly interested in parsimonious models with a low parameter load. A common strategy here is parameter pruning, removing all parameters with small weights. However, here we find this strategy does not work for DSR, where even low magnitude parameters can contribute considerably to the system dynamics. On the other hand, it is well known that many natural systems which generate complex dynamics, like the brain or ecological networks, have a sparse topology with comparatively few links. Inspired by this, we show that geometric pruning, where in contrast to magnitude-based pruning weights with a low contribution to an attractor's geometrical structure are removed, indeed manages to reduce parameter load substantially without significantly hampering DSR quality. We further find that the networks resulting from geometric pruning have a specific type of topology, and that this topology, and not the magnitude of weights, is what is most crucial to performance. We provide an algorithm that automatically generates such topologies which can be used as priors for generative modeling of dynamical systems by RNNs, and compare it to other well studied topologies like small-world or scale-free networks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑