AI 中文总结
本文提出持久传递优化(PDO)方法,通过仅奖励修订后保留的内容来优化流式语音翻译,在FLEURS适配后显著提升BLEU和COMET,并降低延迟与擦除率,且零样本泛化良好。
AI 中文摘要
支持修订的流式语音到文本翻译(S2TT)能够修正早期草稿,但基于可见文本的过程奖励可能会对后来撤回的内容给予奖励。持久传递优化(PDO)仅对在修订中幸存的内容分配中间奖励,同时单独评估最终质量。通过7.49小时的任务特定FLEURS适配,PDO在五个方向中的四个方向上取得了最佳BLEU分数,并且在所有五个方向上的COMET分数均高于每个外部流式基线。相对于其History-SFT初始化,PDO将平均/第90百分位的最终化感知延迟降低了10.8%/11.3%,将归一化擦除率降低了15.8%,同时在第一个允许的2秒更新时输出,并提高了宏观BLEU。在Europarl-ST和CoVoST 2上的零样本评估证实,这些收益并不仅限于FLEURS训练领域。
英文摘要
Revision-capable streaming speech-to-text translation (S2TT) can correct earlier drafts, but process rewards based on visible text may credit content later withdrawn. Persistent Delivery Optimization (PDO) assigns intermediate reward only to content that survives revisions while scoring final quality separately. With 7.49 h of task-specific FLEURS adaptation, PDO achieves the best BLEU on four of five directions and higher COMET than every external streaming baseline in all five directions. Relative to its History-SFT initialization, PDO reduces mean/P90 finalization-aware latency by 10.8\%/11.3\% and normalized erasure by 15.8\%, while emitting at the first permitted 2-s update and improving macro BLEU. Zero-shot evaluation on Europarl-ST and CoVoST 2 confirms that these gains are not confined to the FLEURS training domain.
Comments5 pages, 1 figure, 4 tables. Training code and model weights are available at https://github.com/ggiggit/PDO_S2TT. Submitted to ICASSP 2027