arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.01461cs.LGmath.DGstat.ML
收藏

流形假设下神经网络学习的困难性

Hardness of Learning Neural Networks under the Manifold Hypothesis

John A. Paulson School of Engineering and Applied Sciences, Harvard University(哈佛大学约翰·A·保尔森工程与应用科学学院);Harvard College, Harvard University(哈佛大学哈佛学院)

Bobak T. Kiani, Jason Wang, Melanie Weber

AI总结:

本文研究了流形假设下神经网络学习的困难性,证明了有界曲率输入流形下学习困难,而对流形体积的额外假设可保证可学习性,并探讨了现实数据的中间机制。

摘要:

流形假设假定高维数据位于或接近低维流形。尽管编码几何结构的有效性已在经验上得到证明,但对其在神经网络可学习性影响方面的严格分析在很大程度上仍然缺失。最近的一些结果已经针对 i.i.d. 高斯或均匀布尔数据分布,确立了前馈和等变神经网络的学习困难性结果。在本文中,我们研究了流形假设下学习的困难性。我们探讨对流形的曲率和正则性的哪些最小假设(如果有的话)能够使学习问题变得高效可学习。我们证明,通过将布尔数据输入在 SQ 和密码学设置中的困难性证明扩展到几何设置,在具有有界曲率的输入流形下学习是困难的。另一方面,我们表明,对数据流形体积的额外假设可以缓解这些基本限制,并通过简单的插值论证保证可学习性。该机制的一个显著实例是可以通过流形学习可靠重建的流形。展望未来,我们对具有现实世界数据中常见的异构特征的流形中间机制进行了评论和经验性探索。

英文摘要:

The manifold hypothesis presumes that high-dimensional data lies on or near a low-dimensional manifold. While the utility of encoding geometric structure has been demonstrated empirically, rigorous analysis of its impact on the learnability of neural networks is largely missing. Several recent results have established hardness results for learning feedforward and equivariant neural networks under i.i.d. Gaussian or uniform Boolean data distributions. In this paper, we investigate the hardness of learning under the manifold hypothesis. We ask which minimal assumptions on the curvature and regularity of the manifold, if any, render the learning problem efficiently learnable. We prove that learning is hard under input manifolds of bounded curvature by extending proofs of hardness in the SQ and cryptographic settings for Boolean data inputs to the geometric setting. On the other hand, we show that additional assumptions on the volume of the data manifold alleviate these fundamental limitations and guarantee learnability via a simple interpolation argument. Notable instances of this regime are manifolds which can be reliably reconstructed via manifold learning. Looking forward, we comment on and empirically explore intermediate regimes of manifolds, which have heterogeneous features commonly found in real world data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑