AI 中文总结
针对无偏项两层ReLU神经网络的泊松方程，本研究推导了DNTK矩阵极端特征值的显式上下界，是DNTK谱研究的首批成果，拓展了神经正切核的相关结论。
AI 中文摘要
由微分神经正切核（DNTK）诱导的格拉姆矩阵的最小与最大特征值，在分析通过梯度类算法训练的过参数化物理信息神经网络（PINNs）中发挥关键作用。然而，由于存在多个微分算子带来的挑战，目前完全缺乏对这些极端特征值的理论分析。在本研究中，针对无偏项的两层ReLU神经网络、具有狄利克雷边界条件的泊松方程，我们给出了无限维DNTK矩阵极端特征值的显式上下界。该设置对采样点具有相当的通用性，且在推导最小特征值下界时要求输入维度d≥3。这些结果拓展了神经正切核的相关结论，据我们所知，这是关于DNTK谱的首批研究成果。
英文摘要
The smallest and largest eigenvalues of the Gram matrix induced by the differential neural tangent kernel (DNTK) play a pivotal role in the analysis of over-parameterized PINNs trained by gradient type algorithms. However, a theoretical analysis of the extreme eigenvalues remains completely absent due to the challenge posed by the presence of multiple differential operators. In this work, we provide explicit lower and upper bounds for the extreme eigenvalues of the infinite DNTK matrix for the Poisson equation with the Dirichlet boundary condition for two-layer RePU neural networks without the bias term. The setting is fairly general with respect to the sampling points and input dimension \(d\): \(δ\)-separated and additionally \(d\geq 3\) when deriving the lower bound of the smallest eigenvalue. These results extend that for the neural tangent kernel, and to the best of our knowledge, represent the first results on the spectrum of the DNTK.
Comments19 pages
