arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.11409cs.LGcs.AIeess.SPnlin.CDq-bio.NC
收藏

用于输入重构的回声状态网络无监督学习

Unsupervised Learning in Echo State Networks for Input Reconstruction

Graduate School of Information Science and Technology, The University of Tokyo(东京大学信息科学与技术研究生院);School of Systems Information Science, Future University Hakodate(函馆未来大学系统信息科学系)

Taiki Yamada, Yuichi Katori, Kantaro Fujiwara

AI总结:

本研究提出利用回声状态网络的先验参数及可逆性条件,通过无监督学习实现输入重构,降低对监督的依赖,相关算法适用于自主处理,为理解ESN数学基础及脑计算机制提供新视角。

摘要:

回声状态网络(Echo State Networks,ESNs)是一类仅输出层可训练、循环层与输入层固定的循环神经网络。这种架构约束使其能够对时序数据进行计算高效的处理。传统上,ESN的输出层采用有监督学习结合目标输出进行训练。本研究聚焦于输入重构(Input Reconstruction,IR)任务,即训练输出层以重构输入ESN的输入时序数据。我们证明,只要ESN参数是先验已知的且满足可逆性条件,无需访问有监督目标即可通过无监督学习(Unsupervised Learning,UL)实现输入重构。该公式使得依赖输入重构的应用(如动力系统复制与噪声滤波)可通过与现有算法直接集成,在无监督学习框架下重新构建。我们的结果表明,ESN参数的先验知识可降低对监督的依赖,从而确立了一项新原则:不仅可通过固定部分网络参数,还可通过利用其具体取值来实现。此外,我们提出的用于输入重构及相关任务的基于无监督学习的算法适用于自主处理,为类似计算机制原则上如何在大脑中运作提供了见解。这些发现有助于更深入地理解ESN的数学基础及其与计算神经科学模型的相关性。

英文摘要:

Echo state networks (ESNs) are a class of recurrent neural networks in which only the readout layer is trainable, while the recurrent and input layers are fixed. This architectural constraint enables computationally efficient processing of time-series data. Traditionally, the readout layer in ESNs is trained using supervised learning with target outputs. In this study, we focus on input reconstruction (IR), where the readout layer is trained to reconstruct the input time series fed into the ESN. We show that IR can be achieved through unsupervised learning (UL), without access to supervised targets, provided that the ESN parameters are known a priori and satisfy invertibility conditions. This formulation allows applications relying on IR, such as dynamical system replication and noise filtering, to be reformulated within the UL framework via straightforward integration with existing algorithms. Our results suggest that prior knowledge of ESN parameters can reduce reliance on supervision, thereby establishing a new principle: not only by fixing part of the network parameters but also by exploiting their specific values. Furthermore, our UL-based algorithms for input reconstruction and related tasks are suitable for autonomous processing, offering insights into how analogous computational mechanisms might operate in the brain in principle. These findings contribute to a deeper understanding of the mathematical foundations of ESNs and their relevance to models in computational neuroscience.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑