深度ReLU表示的布朗头：激活质量与同样本选择的代价

中文摘要 AI 辅助

深度表示学习通常在同一个样本上选择隐藏特征并拟合最终预测器，因此在选择后进行的固定特征分析可能会忽略选择代价。我们研究了在加法或Lévy-布朗再生核希尔伯特空间（RKHS）中，由有界范数预测器跟随的深度ReLU表示的条件经验Rademacher复杂度，这些空间被称为布朗头。对于固定表示，我们推导出一个精确的对偶恒等式，并以激活质量（观测到的隐藏向量的平均范数）给出了尖锐界。在同样本选择下，表示的上确界诱导出一个二次Rademacher过程。布朗分层蛋糕和高斯投影恒等式将其简化为逐坐标或带符号的投影阈值迹，从而将实现尺度与选择复杂度分开。对于具有成对不同输入的样本，显式标量ReLU族在实现迹和包络水平上，与有限迹和VC速率匹配至通用常数。诱导范数收缩也为矩形、秩亏的ReLU网络提供了架构级界。实验验证了尖锐界和速率，展示了在固定激活质量下的选择差距，并评估了布朗头的预测可行性。

英文摘要

Deep representation learning often selects hidden features and fits the final predictor on the same sample, so fixed-feature analysis performed after selection can omit selection cost. We study the conditional empirical Rademacher complexity of deep ReLU representations followed by bounded-norm predictors in additive or Lévy-Brownian RKHSs, termed Brownian heads. For a fixed representation, we derive an exact dual identity and sharp bounds in terms of activation mass, the average norm of the observed hidden vectors. Under same-sample selection, the representation supremum induces a quadratic Rademacher process. Brownian layer-cake and Gaussian-projection identities reduce it to coordinatewise or signed projected threshold traces, separating realized scale from selection complexity. For samples with pairwise-distinct inputs, explicit scalar ReLU families match the finite-trace and VC rates up to universal constants at the realized trace-and-envelope level. Induced-norm contraction also yields architecture-level bounds for rectangular, rank-deficient ReLU networks. Experiments verify the sharp bounds and rates, exhibit a selection gap at fixed activation mass, and assess the predictive feasibility of Brownian heads.

发表机构

• Free University of Bozen–Bolzano（博尔扎诺自由大学）
• Technische Universität Braunschweig（布伦瑞克工业大学）

机构由 AI 辅助整理，请以论文原文为准。