基于商特征空间的群不变/等变深度网络泛化界的改进
Improved Generalization Bounds of Group Invariant / Equivariant Deep Networks via Quotient Feature Spaces
摘要：
许多不变（或等变）神经网络已经成功地处理不变的数据，如点云和图形。然而，神经网络的推广理论还没有得到很好的发展，因为该理论的几个基本因素，如网络大小和边缘分布，没有深入联系的不变性和等变性。在这项研究中，我们开发了一个新的泛化误差界不变和等变深度神经网络。为了描述不变性和等变性对推广的影响，我们提出了一个\textit{quotient feature space}的概念，它度量了群作用对性质的影响。我们的主要结果证明了商特征空间的体积可以描述泛化误差。此外，该界表明，不变性和等变性显着改善的领先项的界。我们将我们的结果应用于特定的不变和等变网络，例如DeepSets（Zaheer et al.（2017）），并表明它们的泛化界被$\sqrt{n!}$大大提高，其中$n!$是排列的数量。我们还讨论了不变DNN的表达能力，并表明它们可以实现最佳近似率。我们的实验结果支持了我们的理论主张。
英文摘要：
Numerous invariant (or equivariant) neural networks have succeeded in handling invariant data such as point clouds and graphs. However, a generalization theory for the neural networks has not been well developed, because several essential factors for the theory, such as network size and margin distribution, are not deeply connected to the invariance and equivariance. In this study, we develop a novel generalization error bound for invariant and equivariant deep neural networks. To describe the effect of invariance and equivariance on generalization, we develop a notion of a \textit{quotient feature space}, which measures the effect of group actions for the properties. Our main result proves that the volume of quotient feature spaces can describe the generalization error. Furthermore, the bound shows that the invariance and equivariance significantly improve the leading term of the bound. We apply our result to specific invariant and equivariant networks, such as DeepSets (Zaheer et al. (2017)), and show that their generalization bound is considerably improved by $\sqrt{n!}$, where $n!$ is the number of permutations. We also discuss the expressive power of invariant DNNs and show that they can achieve an optimal approximation rate. Our experimental result supports our theoretical claims.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
