用于高效多语言机器翻译和语音翻译的Transformer-编码树
Transformer-Encoder Trees for Efficient Multilingual Machine Translation and Speech Translation
AI总结：
本文提出Transformer-编码树(TET)，通过共享相似语言的中间表示提升低资源语言准确率，减少计算冗余，实现单次前向传递生成所有目标语言，比传统多语言设计减少66%参数和60%推理计算。
摘要：
多语言翻译面临计算冗余问题，尤其在同时翻译多种语言时更为明显。此外，低资源语言的翻译质量也面临挑战。为此，我们引入Transformer编码树(TET)，一种层次化、非自回归的编码器-only架构，通过连接主义时序分类(CTC)进行训练，用于多语言翻译。TET在语言学相似的目标语言间共享中间表示，从而在低资源语言上提高准确性，同时减少计算冗余并允许在单次前向传递中生成所有目标语言。TET消除了自回归模型的序列瓶颈，支持所有目标语言上所有token的完全并行解码。与传统多语言设计相比，TET将总参数数量减少了66%，推理计算降低了60%。在语音翻译中，将TET与非自回归语音识别骨干网络(Wav2Vec2)结合，显示出与自回归系统相媲美的翻译质量，同时推理速度提高了约7-14倍。
英文摘要：
Multilingual translation suffers from computational redundancy, especially when translating into multiple languages simultaneously. In addition, translation quality can suffer for low-resource languages. To address this, we introduce Transformer Encoder Tree (TET), a hierarchical, non-autoregressive encoder-only architecture trained with Connectionist Temporal Classification (CTC) for multilingual translation. TET shares intermediate representations among linguistically similar target languages, improving accuracy on low-resource languages while reducing computational redundancy and enabling the generation of all target languages in a single forward pass. TET eliminates the sequential bottleneck of autoregressive models and supports fully parallel decoding of all tokens across all target languages. Compared to a naive one-to-many multilingual design, TET reduces the total parameter count by 66% and lowers inference computation by 60%. In speech translation, combining TET with a non-autoregressive speech recognition backbone (Wav2Vec2) shows competitive translation quality compared to autoregressive systems while speeding up inference by approximately 7-14 times.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
