SimulMT to SimulST: Adapting Simultaneous Text Translation to End-to-End Simultaneous Speech Translation

Decision Attentive Regularization to Improve Simultaneous Speech Translation Systems

Simultaneous translation systems start producing the output while processing the partial source sentence in the incoming input stream. These systems need to decide when to read more input and when to write the output. These decisions depend on the structure of source/target language and the information contained in the partial input sequence. Hence, read/write decision policy remains the same across different input modalities, i.e., speech and text. This motivates us to leverage the text transcripts corresponding to the speech input for improving simultaneous speech-to-text translation (SimulS…

Simul-LLM：一个探索利用大语言模型进行高质量同声传译的框架

Simul-LLM: A Framework for Exploring High-Quality Simultaneous Translation with Large Language Models

本文提出Simul-LLM框架，研究将大语言模型应用于同声传译任务，验证经典SimulMT方法并实现开源微调与评估流水线。

Exploring Continuous Integrate-and-Fire for Adaptive Simultaneous Speech Translation

Simultaneous speech translation (SimulST) is a challenging task aiming to translate streaming speech before the complete input is observed. A SimulST system generally includes two components: the pre-decision that aggregates the speech information and the policy that decides to read or write. While recent works had proposed various strategies to improve the pre-decision, they mainly adopt the fixed wait-k policy, leaving the adaptive policies rarely explored. This paper proposes to model the adaptive policy by adapting the Continuous Integrate-and-Fire (CIF). Compared with monotonic multihead …