隐式神经表示中的入口秩崩溃：诊断与统一解决
The Inlet Rank Collapse in Implicit Neural Representations: Diagnosis and Unified Remedy
AI总结：
本文提出了一种结构诊断框架，揭示隐式神经表示中入口秩崩溃现象，并通过秩扩展初始化方法提升模型表现。
摘要：
隐式神经表示（INRs）已经革新了连续信号建模，但它们在有限的训练预算内难以恢复细粒度细节。尽管经验技术，如位置编码（PE）、正弦激活（SIREN）和批量归一化（BN），能够有效缓解这一问题，但它们的理论依据主要是事后解释，仅关注修改后全局NTK谱。在本工作中，我们反转了这一范式，引入了结构诊断框架。通过逐层分解NTK，我们数学上识别出所谓的"入口秩崩溃"：一种现象，即低维输入坐标无法覆盖高维嵌入空间，导致第一层出现根本性的秩不足，成为整个网络的表达瓶颈。该框架提供了一个统一的视角，重新解释PE、SIREN和BN为不同形式的秩恢复。基于此诊断，我们推导出一种秩扩展初始化，这是一种最小化的解决方法，确保表示秩与层宽成比例增长，而无需架构修改或计算开销。我们的结果表明，这种原理性的解决方法使标准MLP能够实现高保真的重建，证明了使INRs发挥作用的关键在于初始秩传播的结构优化，以有效填充潜在空间。
英文摘要：
Implicit Neural Representations (INRs) have revolutionized continuous signal modeling, yet they struggle to recover fine-grained details within finite training budgets. While empirical techniques, such as positional encoding (PE), sinusoidal activations (SIREN), and batch normalization (BN), effectively mitigate this, their theoretical justifications are predominantly post hoc, focusing on the global NTK spectrum only after modifications are applied. In this work, we reverse this paradigm by introducing a structural diagnostic framework. By performing a layer-wise decomposition of the NTK, we mathematically identify the ``Inlet Rank Collapse'': a phenomenon where the low-dimensional input coordinates fail to span the high-dimensional embedding space, creating a fundamental rank deficiency at the first layer that acts as an expressive bottleneck for the entire network. This framework provides a unified perspective to re-interpret PE, SIREN, and BN as different forms of rank restoration. Guided by this diagnosis, we derive a Rank-Expanding Initialization, a minimalist remedy that ensures the representation rank scales with the layer width without architectural modifications or computational overhead. Our results demonstrate that this principled remedy enables standard MLPs to achieve high-fidelity reconstructions, proving that the key to empowering INRs lies in the structural optimization of the initial rank propagation to effectively populate the latent space.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
