arXiv 2006.134012020-06cs.LGstat.ML

理解具有推理层的深层架构

Understanding Deep Architectures with Reasoning Layer

Georgia Institute of Technology（佐治亚理工学院）；University of Oxford（牛津大学）

Xinshi Chen, Yufei Zhang, Christoph Reisinger, Le Song

备注：: 34th Conference on Neural Information Processing Systems (NeurIPS 2020)

摘要：

最近，人们对将深度学习模型与推理相结合以处理更复杂的学习任务的兴趣激增。在许多情况下，推理任务可以通过迭代算法来解决。该算法通常被展开，并用作深度架构中的专用层，可以与其他神经组件进行端到端训练。虽然这种混合深度架构已经取得了许多经验上的成功，但这种架构的理论基础，特别是算法层和其他神经层之间的相互作用，在很大程度上仍然没有得到探索。在本文中，我们通过展示算法层的属性（如收敛性、稳定性和灵敏度）与端到端模型的近似和泛化能力密切相关，从而迈出了理解这种混合深度架构的第一步。此外，我们的分析与我们在各种条件下的实验观察结果非常吻合，这表明我们的理论可以为设计具有推理层的深层架构提供有用的指导。

英文摘要：

Recently, there has been a surge of interest in combining deep learning models with reasoning in order to handle more sophisticated learning tasks. In many cases, a reasoning task can be solved by an iterative algorithm. This algorithm is often unrolled, and used as a specialized layer in the deep architecture, which can be trained end-to-end with other neural components. Although such hybrid deep architectures have led to many empirical successes, the theoretical foundation of such architectures, especially the interplay between algorithm layers and other neural layers, remains largely unexplored. In this paper, we take an initial step towards an understanding of such hybrid deep architectures by showing that properties of the algorithm layers, such as convergence, stability, and sensitivity, are intimately related to the approximation and generalization abilities of the end-to-end model. Furthermore, our analysis matches closely our experimental observations under various conditions, suggesting that our theory can provide useful guidelines for designing deep architectures with reasoning layers.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

摘要：

英文摘要：

相关论文