什么使数据适合局部连接神经网络?一个基于量子纠缠的充要条件
What Makes Data Suitable for a Locally Connected Neural Network? A Necessary and Sufficient Condition Based on Quantum Entanglement
AI总结:
本文利用量子纠缠刻画数据分布与局部连接神经网络的适配性,证明低纠缠是准确预测的充要条件,并提出可提升适配性的预处理方法。
摘要:
何种因素使数据分布适合深度学习,是一个根本性的开放问题。本文聚焦局部连接神经网络——一类包含卷积神经网络、循环神经网络以及局部自注意力模型的主流架构——并借助量子物理学中的理论工具来解决该问题。我们的主要理论结果表明:某个局部连接神经网络能够对某一数据分布进行准确预测,当且仅当该数据分布在特征的某些规范划分下具有较低的量子纠缠。作为这一结果的实际应用,我们推导出一种预处理方法,用于提升数据分布对局部连接神经网络的适配性。在多种数据集上使用广泛模型开展的实验验证了我们的发现。我们希望,本文对量子纠缠的运用将鼓励研究者进一步采用物理学工具,对深度学习与现实世界数据之间的关系进行形式化推理。
英文摘要:
The question of what makes a data distribution suitable for deep learning is a fundamental open problem. Focusing on locally connected neural networks (a prevalent family of architectures that includes convolutional and recurrent neural networks as well as local self-attention models), we address this problem by adopting theoretical tools from quantum physics. Our main theoretical result states that a certain locally connected neural network is capable of accurate prediction over a data distribution if and only if the data distribution admits low quantum entanglement under certain canonical partitions of features. As a practical application of this result, we derive a preprocessing method for enhancing the suitability of a data distribution to locally connected neural networks. Experiments with widespread models over various datasets demonstrate our findings. We hope that our use of quantum entanglement will encourage further adoption of tools from physics for formally reasoning about the relation between deep learning and real-world data.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。