随机初始化自编码器：不动点与混沌边缘
Randomly initialized autoencoders: fixed points and edge-of-chaos
AI总结：
本文研究随机初始化自编码器的不动点问题，引入针对自编码器的局部与全局混沌边缘概念，结合随机矩阵理论与高斯过程不等式开展分析，为自编码器的稳定性研究提供理论支撑。
AI中文摘要：
本文研究自编码器，这是一类特殊的深度神经网络（DNNs），其性能可通过不动点表征。该视角自然引出了这些不动点的存在性、稳定性及吸引域问题，这些问题通过自编码器的压缩性性质解决，且与混沌边缘（Edge-of-Chaos, EoC）概念密切相关。EoC是DNN理论中的重要概念，描述了随机初始化网络中有序与混沌信号传播的临界状态，在该临界状态或附近初始化具有网络对输入扰动的稳定性等理论与实践优势，此前已通过平均场平均法针对广泛类别的神经网络引入。本文将EoC概念修改以用于自编码器研究，具体引入自编码器的局部EoC与全局EoC，分别控制输入的局部（小）扰动与全局（任意）扰动。自编码器的稳定性研究属于随机矩阵理论（Random Matrix Theory, RMT）的非线性问题范畴，本文对局部EoC的分析基于RMT的谱技术，而全局EoC的研究则采用高斯过程的Sudakov-Fernique不等式。
英文摘要：
In this paper we study autoencoders, a special class of deep neural nets (DNNs) whose performance can be characterized via their fixed points. This perspective naturally raises questions of existence, stability, and basins of attraction of these fixed points. These questions are addressed via the contractive properties of autoencoders, and are closely related to the notion of edge-of-chaos. Edge-of-chaos (EoC) is an important notion in the theory of DNNs. It describes the critical regime separating ordered and chaotic signal propagation through a randomly initialized network. Initialization at or near this critical regime offers several theoretical and practical advantages, including stability of the network w.r.t. perturbations of the input. EoC was previously introduced for broad classes of neural networks using mean-field averaging methods. In this paper we modify the notion of EoC for the study of autoencoders. Specifically, we introduce local and global EoC for autoencoders that control local (small) and global (arbitrary) perturbations of the input respectively. The study of stability of autoencoders falls within the scope of nonlinear problems in Random Matrix Theory (RMT). Our analysis of local EoC is based on spectral techniques of RMT, whereas global EoC is studied by employing Sudakov-Fernique inequality for Gaussian processes.
