arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.14966cs.LGcs.AIcs.ITmath.IT
收藏

纠错码Transformer

Error Correction Code Transformer

Tel Aviv University(特拉维夫大学)

Yoni Choukroun, Lior Wolf

摘要:

纠错码是通信物理层的主要组成部分,确保数据在噪声信道上的可靠传输。最近,神经解码器被证明优于经典解码技术。然而,现有的神经方法由于指数训练复杂性而呈现出强烈的过拟合,或者由于依赖于置信传播而呈现出限制性的归纳偏差。最近,由于Transformers能够表示元素之间的复杂交互,因此它已成为许多应用程序中的首选方法。在这项工作中,我们提出了第一次扩展的Transformer架构的线性码在任意块长度的软解码。我们将每个通道的输出维度编码为高维,以便更好地表示要单独处理的比特信息。逐元素处理允许信道输出可靠性的分析,而代数码和位之间的相互作用通过自适应掩蔽自注意模块插入到模型中。所提出的方法展示了Transformers的强大功能和灵活性,并在时间复杂度的一小部分上大幅优于现有的最先进的神经解码器。

英文摘要:

Error correction code is a major part of the communication physical layer, ensuring the reliable transfer of data over noisy channels. Recently, neural decoders were shown to outperform classical decoding techniques. However, the existing neural approaches present strong overfitting due to the exponential training complexity, or a restrictive inductive bias due to reliance on Belief Propagation. Recently, Transformers have become methods of choice in many applications thanks to their ability to represent complex interactions between elements. In this work, we propose to extend for the first time the Transformer architecture to the soft decoding of linear codes at arbitrary block lengths. We encode each channel's output dimension to high dimension for better representation of the bits information to be processed separately. The element-wise processing allows the analysis of the channel output reliability, while the algebraic code and the interaction between the bits are inserted into the model via an adapted masked self-attention module. The proposed approach demonstrates the extreme power and flexibility of Transformers and outperforms existing state-of-the-art neural decoders by large margins at a fraction of their time complexity.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑