神经机器翻译中的依赖可分离卷积
Depthwise Separable Convolutions for Neural Machine Translation
摘要：
依赖可分离卷积减少了卷积运算中使用的参数和计算的数量，同时提高了表示效率。他们已经被证明是成功的图像分类模型，无论是在获得更好的模型比以前可能为一个给定的参数计数（Xception架构），并大大减少了参数的数量需要在一个给定的水平（MobileNets系列的架构）。最近，卷积序列到序列网络已被应用于机器翻译任务，并取得了良好的效果。在这项工作中，我们研究如何dependently可分离卷积可以应用于神经机器翻译。我们引入了一种受Xception和ByteNet启发的新架构，称为SliceNet，它可以显着减少获得ByteNet等结果所需的参数计数和计算量，并且使用类似的参数计数，实现了新的最先进的结果。除了证明dependency可分离卷积在机器翻译中表现良好外，我们还研究了它们所带来的架构变化：我们观察到，由于dependency可分离性，我们可以增加卷积窗口的长度，从而消除对滤波器膨胀的需要。我们还引入了一个新的“超可分离”卷积运算，进一步减少了参数的数量和计算成本，以获得最先进的结果。
英文摘要：
Depthwise separable convolutions reduce the number of parameters and computation used in convolutional operations while increasing representational efficiency. They have been shown to be successful in image classification models, both in obtaining better models than previously possible for a given parameter count (the Xception architecture) and considerably reducing the number of parameters required to perform at a given level (the MobileNets family of architectures). Recently, convolutional sequence-to-sequence networks have been applied to machine translation tasks with good results. In this work, we study how depthwise separable convolutions can be applied to neural machine translation. We introduce a new architecture inspired by Xception and ByteNet, called SliceNet, which enables a significant reduction of the parameter count and amount of computation needed to obtain results like ByteNet, and, with a similar parameter count, achieves new state-of-the-art results. In addition to showing that depthwise separable convolutions perform well for machine translation, we investigate the architectural changes that they enable: we observe that thanks to depthwise separability, we can increase the length of convolution windows, removing the need for filter dilation. We also introduce a new "super-separable" convolution operation that further reduces the number of parameters and computational cost for obtaining state-of-the-art results.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
