用于硬2D任务的矩阵混洗交换网络
Matrix Shuffle-Exchange Networks for Hard 2D Tasks
摘要:
卷积神经网络已成为处理二维数据的主要工具。它们对图像效果很好,但卷积的接受域有限,这阻碍了它在更复杂的2D任务中的应用。我们提出了一种新的神经模型,称为矩阵洗牌交换网络,可以有效地利用2D数据中的长程依赖关系,并具有与卷积神经网络相当的速度。它源自神经混洗-交换网络,并且具有$\mathcal{O}( \log{n})$层和用于处理$n \times n$数据矩阵的$\mathcal{O}( n^2 \log{n})$总时间和空间复杂度。我们表明,矩阵洗牌交换网络非常适合矩阵和密集图上的算法和逻辑推理任务,超过了卷积和图神经网络的基线。它的明显优势是,当推广到更大的实例时,能够保持完整的长距离依赖建模--比配备密集注意力机制的模型所能处理的要大得多。
英文摘要:
Convolutional neural networks have become the main tools for processing two-dimensional data. They work well for images, yet convolutions have a limited receptive field that prevents its applications to more complex 2D tasks. We propose a new neural model, called Matrix Shuffle-Exchange network, that can efficiently exploit long-range dependencies in 2D data and has comparable speed to a convolutional neural network. It is derived from Neural Shuffle-Exchange network and has $\mathcal{O}( \log{n})$ layers and $\mathcal{O}( n^2 \log{n})$ total time and space complexity for processing a $n \times n$ data matrix. We show that the Matrix Shuffle-Exchange network is well-suited for algorithmic and logical reasoning tasks on matrices and dense graphs, exceeding convolutional and graph neural network baselines. Its distinct advantage is the capability of retaining full long-range dependency modelling when generalizing to larger instances - much larger than could be processed with models equipped with a dense attention mechanism.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。