深度可实现局部熵：深度变分范数ReLU回归中的二次深度依赖

中文摘要 AI 辅助

我们研究了基于显式向量值Parhi–Nowak深度-RBV²架构的高斯回归，该架构具有深度L、宽度w、层和变分预算A以及输出边界B。对于这个参数规模为O(L w²)的架构，已知的下界和上界相差一个深度因子。我们构造了一个局部填充，表明在明确的样本量依赖半径条件下，二次深度依赖是内在的。该填充的对数基数为Ω(L² w² log w)；其码字位于O(λ) L²球内，且两两之间的距离为Ω(λ)。主要组成部分是偏差校正的有界系数逼近定理和平衡放大：将深度D的ReLU网络乘以q可通过一个恒定通道实现，使得每个系数仅增长q^(1/D)。转换为向量值RBV²块后，层和成本为O(D w² q^(1/D))。高斯Fano不等式给出了由输出、测试和表示尺度决定的显式半径下界。在A=B=R、σ与R成比例以及上述半径条件下，这给出了极小极大风险至少为L² w² log(w) R²/n量级。基于伪维度的有限网上界给出了无界高斯响应下的Õ(L² w² R²/n)。因此，极小极大风险对深度具有二次多项式依赖，直至对数因子，并在较小半径处表现出向表示受限行为的转变。

英文摘要

We study Gaussian regression over the explicit vector-valued Parhi--Nowak deep-RBV^2 architecture with depth L, width w, layer-sum variation budget A, and output bound B. For this O(L w^2)-parameterized architecture, the known lower and upper bounds differ by one factor of depth. We construct a local packing showing that the quadratic depth dependence is intrinsic under an explicit sample-size-dependent radius condition. The packing has log-cardinality Omega(L^2 w^2 log w); its codewords lie in an O(lambda) L^2 ball and are pairwise Omega(lambda)-separated. The main ingredients are a bias-corrected bounded-coefficient approximation theorem and balanced amplification: multiplying a depth-D ReLU network by q can be implemented using one constant channel so that every coefficient grows by only q^(1/D). Translation to vector-valued RBV^2 blocks then has layer-sum cost O(D w^2 q^(1/D)). Gaussian Fano yields a radius-explicit lower bound governed by the output, testing, and representation scales. Under A=B=R, sigma proportional to R, and the stated radius condition, this gives minimax risk at least of order L^2 w^2 log(w) R^2/n. A pseudodimension-based finite-net upper bound gives O-tilde(L^2 w^2 R^2/n) for unbounded Gaussian responses. Thus the minimax risk has quadratic polynomial dependence on depth, up to logarithmic factors, and exhibits a transition to representation-limited behavior at smaller radius.

发表机构

• Key Laboratory of System Software (Chinese Academy of Sciences)（系统软件重点实验室（中国科学院））
• State Key Laboratory of Computer Science（计算机科学国家重点实验室）
• Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）
• School of Computer Science and Technology, University of Chinese Academy of Sciences（中国科学院大学计算机科学与技术学院）

机构由 AI 辅助整理，请以论文原文为准。