位置编码在基于Transformer的单声道语音增强中影响的实证研究
An Empirical Study on the Impact of Positional Encoding in Transformer-based Monaural Speech Enhancement
AI总结：
本文实证研究了五种位置编码方法在因果与非因果配置下对基于Transformer的单声道语音增强的影响，发现非因果配置显著受益于位置编码且相对位置嵌入优于绝对位置嵌入，而因果配置则帮助不大。
摘要：
Transformer架构推动了语音增强领域的近期进展。由于Transformer是位置无关的，位置编码是使其能够区分序列中元素顺序的事实标准组件。然而，位置编码究竟如何影响基于Transformer架构的语音增强仍不明确。本文进行了一项全面的实证研究，在因果和非因果配置下评估了五种位置编码方法，即正弦和可学习的绝对位置嵌入（APE）、T5-RPE、KERPLE，以及无位置编码的Transformer（No-Pos）。我们进行了广泛的语音增强实验，涉及频谱映射和掩蔽方法。研究结果表明，在因果配置中，位置编码对模型帮助不大，这表明因果注意力可能隐式包含了位置信息。在非因果配置中，模型显著受益于位置编码的使用。此外，我们发现在这四种位置嵌入中，相对位置嵌入优于APE。
英文摘要：
Transformer architecture has enabled recent progress in speech enhancement. Since Transformers are position-agostic, positional encoding is the de facto standard component used to enable Transformers to distinguish the order of elements in a sequence. However, it remains unclear how positional encoding exactly impacts speech enhancement based on Transformer architectures. In this paper, we perform a comprehensive empirical study evaluating five positional encoding methods, i.e., Sinusoidal and learned absolute position embedding (APE), T5-RPE, KERPLE, as well as the Transformer without positional encoding (No-Pos), across both causal and noncausal configurations. We conduct extensive speech enhancement experiments, involving spectral mapping and masking methods. Our findings establish that positional encoding is not quite helpful for the models in a causal configuration, which indicates that causal attention may implicitly incorporate position information. In a noncausal configuration, the models significantly benefit from the use of positional encoding. In addition, we find that among the four position embeddings, relative position embeddings outperform APEs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
