多项式复合的线性独立性与深度神经网络的可识别性

中文摘要 AI 辅助

受深度学习理论问题的启发，我们提出猜想：将固定数量的两两不同的非恒定多项式与一个足够大次数的泛多项式复合后，得到的多项式是线性独立的，这一猜想推广了Newman–Slater关于多项式幂的定理。我们证明了该猜想及其“原点传递”变体的若干情形：证明了两个多项式的情形，以及当多项式次数有界时任意数量多项式的情形。此外，我们展示了该猜想如何帮助完全理解具有泛多项式激活函数的深度全连接神经网络架构的可识别性（即参数对称性）。特别地，对于具有递增次数的层特定激活函数的网络架构，我们证明的猜想版本完全刻画了产生相同端到端网络函数的参数集合；作为特例，我们完全解决了浅层多项式网络的可识别性问题。

英文摘要

Motivated by theoretical problems in deep learning, we conjecture that post-composing a fixed number of pairwise distinct nonconstant polynomials with a generic polynomial of sufficiently large degree yields linearly independent polynomials. This generalizes Newman--Slater's theorem on powers of polynomials. We establish several cases of this conjecture and its origin-passing variant: We prove the result for two polynomials, and for an arbitrary number of polynomials when their degrees are bounded. Furthermore, we show how the conjecture implies a complete understanding of the identifiability (i.e., parameter symmetries) of deep fully connected neural network architectures with generic polynomial activation functions. In particular, for network architectures with layer-specific activations of increasing degree, our established versions of the conjecture fully characterize the set of parameters yielding the same end-to-end network function. As a special case, we fully resolve the identifiability of shallow polynomial networks.

发表机构

• KTH Stockholm（瑞典皇家理工学院）
• Digital Futures（数字未来机构）
• AI4S AB Stockholm（AI4S AB 斯德哥尔摩公司）
• University of Ferrara（费拉拉大学）

机构由 AI 辅助整理，请以论文原文为准。