AI 中文总结
该研究探讨初始化时各层权重相关的深度残差网络的深层行为，证实并推广了Marion等人的猜想，确定了临界缩放与渐近极限的决定因素，建立了巴拿赫空间中杨微分方程的鲁棒稳定性理论。
AI 中文摘要
我们研究了初始化时各层权重存在相关性的深度残差网络的深层行为。我们的结果证实并推广了Marion等人[2025]的猜想，该猜想指出，相关初始化应能在独立初始化产生的布朗随机微分方程与完全相关初始化产生的常微分方程之间连续插值。当初始化由对具有正则变化相关性的平稳高斯序列应用特征函数得到时，我们证明存在唯一的临界缩放，使得无限深度极限是由厄米特过程驱动的杨微分方程的解。若生成初始化的特征函数的厄米特秩为1（例如恒等函数的情况），则厄米特过程会退化为分数布朗运动。我们表明，临界缩放和渐近极限由相关性衰减以及特征函数的厄米特秩唯一确定。因此，在渐近 regime 中，初始化的相关结构和厄米特秩是有意义的超参数。相比之下，在有限方差独立同分布初始化下，无论分布选择如何，渐近驱动项在归一化后均为通用布朗运动。我们的证明依赖于一系列新结果，这些结果建立了巴拿赫空间中杨微分方程的鲁棒稳定性理论。
英文摘要
We study the large-depth behavior of residual networks whose weights are correlated across layers at initialization. Our results confirm and extend a conjecture of Marion et al. [2025], according to which correlated initializations should interpolate continuously between the Brownian stochastic differential equation arising from independent initialization and the ordinary differential equation arising from perfectly correlated initialization. When the initialization is obtained from the application of a feature function to a stationary Gaussian sequence with regularly varying correlation, we prove that there exists a unique critical scaling such that the infinite-depth limit is the solution of a Young differential equation driven by a Hermite process. Hermite processes reduce to the fractional Brownian motion if the feature function generating the initialization has Hermite rank one, which is the case for the identity function, for example. We show that the critical scaling and asymptotic limit are uniquely determined by the decay of correlations together with the Hermite rank of the feature function. Consequently, the correlation structure and Hermite rank of the initialization represent meaningful hyperparameters in the asymptotic regime. By contrast, under finite-variance iid initialization, the asymptotic driver is universally Brownian up to normalization regardless of the choice of distribution. Our proofs rely on a collection of novel results establishing a robust stability theory for Young differential equations in Banach spaces.
