通过 RNN 实现线性时不变（LTI）系统的通用逼近：储备池计算中随机性的力量
Universal Approximation of Linear Time-Invariant (LTI) Systems through RNNs: Power of Randomness in Reservoir Computing
AI总结：
本文从信号处理角度证明储备池计算可通用逼近一般 LTI 系统，并解析得到配置其未训练循环权重的最优概率分布。
摘要：
众所周知，循环神经网络（RNN）在相当温和且一般的假设下是动态系统的通用逼近器。然而，在标准 RNN 训练中，RNN 通常会遭遇梯度消失和梯度爆炸问题。储备池计算（Reservoir computing，RC）是一种特殊的 RNN，其循环权重被随机化且不进行训练；该方法被提出以克服这些问题，并已展现出优越的经验性能，尤其是在训练样本极其有限的场景中。另一方面，支撑这种已观测性能的理论基础尚未得到充分发展。在本工作中，我们证明 RC 能够通用逼近一般的线性时不变（LTI）系统。具体而言，我们给出了对 RC 的清晰信号处理解释，并利用这一理解来解决逼近一般 LTI 系统的问题。在此设置下，我们通过解析刻画了用于配置（而非训练和/或随机生成）RC 底层 RNN 循环权重的最优概率密度函数。我们提供了大量数值评估，以验证所推导分布在配置 RC 循环权重以逼近一般 LTI 系统时的最优性。我们的工作带来了 RC 清晰的、基于信号处理的模型可解释性，并为随机生成而非训练 RC 循环权重时随机性的力量提供了理论解释与依据。此外，它为配置未经训练的循环权重提供了完整的最优解析刻画，标志着面向可解释机器学习（XML）以融合领域知识实现高效学习迈出的重要一步。
英文摘要：
Recurrent neural networks (RNNs) are known to be universal approximators of dynamic systems under fairly mild and general assumptions. However, RNNs usually suffer from the issues of vanishing and exploding gradients in standard RNN training. Reservoir computing (RC), a special RNN where the recurrent weights are randomized and left untrained, has been introduced to overcome these issues and has demonstrated superior empirical performance especially in scenarios where training samples are extremely limited. On the other hand, the theoretical grounding to support this observed performance has yet been fully developed. In this work, we show that RC can universally approximate a general linear time-invariant (LTI) system. Specifically, we present a clear signal processing interpretation of RC and utilize this understanding in the problem of approximating a generic LTI system. Under this setup, we analytically characterize the optimum probability density function for configuring (instead of training and/or randomly generating) the recurrent weights of the underlying RNN of the RC. Extensive numerical evaluations are provided to validate the optimality of the derived distribution for configuring the recurrent weights of the RC to approximate a general LTI system. Our work results in clear signal processing-based model interpretability of RC and provides theoretical explanation/justification for the power of randomness in randomly generating instead of training RC's recurrent weights. Furthermore, it provides a complete optimum analytical characterization for configuring the untrained recurrent weights, marking an important step towards explainable machine learning (XML) to incorporate domain knowledge for efficient learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
