arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2304.04596cs.SDcs.CLeess.AS
收藏

ESPnet-ST-v2:多用途口语翻译工具包

ESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkit

Carnegie Mellon University(卡内基梅隆大学);Meta AI;Charles University(查理大学);Nagoya University(名古屋大学);Johns Hopkins University(约翰斯·霍普金斯大学)
Brian Yan, Jiatong Shi, Yun Tang, Hirofumi Inaguma, Yifan Peng, Siddharth Dalmia, Peter Polák, Patrick Fernandes, Dan Berrebbi, Tomoki Hayashi, Xiaohui Zhang, Z… 展开作者

Brian Yan, Jiatong Shi, Yun Tang, Hirofumi Inaguma, Yifan Peng, Siddharth Dalmia, Peter Polák, Patrick Fernandes, Dan Berrebbi, Tomoki Hayashi, Xiaohui Zhang, Zhaoheng Ni, Moto Hira, Soumi Maiti, Juan Pino, Shinji Watanabe

AI总结:

ESPnet-ST-v2 是一个多用途开源口语翻译工具包,支持离线语音到文本翻译、同声传译和语音到语音翻译,并提供多种最先进架构与性能基准。

摘要:

ESPnet-ST-v2 是对开源 ESPnet-ST 工具包的一次全面革新,这一革新源于口语翻译社区日益广泛的兴趣。ESPnet-ST-v2 支持 1) 离线语音到文本翻译(ST)、2) 同声传译语音到文本翻译(SST)以及 3) 离线语音到语音翻译(S2ST)——每项任务都得到了多种方法的支持,这使得 ESPnet-ST-v2 与其他开源口语翻译工具包区别开来。该工具包提供了最先进的架构,例如 transducer、混合 CTC/注意力、具有可搜索中间表示的多解码器、时间同步分块 CTC/注意力、Translatotron 模型以及直接离散单元模型。在本文中,我们描述了 ESPnet-ST-v2 背后的整体设计、每项任务的示例模型以及性能基准测试,该工具包已在 https://github.com/espnet/espnet 上公开发布。

英文摘要:

ESPnet-ST-v2 is a revamp of the open-source ESPnet-ST toolkit necessitated by the broadening interests of the spoken language translation community. ESPnet-ST-v2 supports 1) offline speech-to-text translation (ST), 2) simultaneous speech-to-text translation (SST), and 3) offline speech-to-speech translation (S2ST) -- each task is supported with a wide variety of approaches, differentiating ESPnet-ST-v2 from other open source spoken language translation toolkits. This toolkit offers state-of-the-art architectures such as transducers, hybrid CTC/attention, multi-decoders with searchable intermediates, time-synchronous blockwise CTC/attention, Translatotron models, and direct discrete unit models. In this paper, we describe the overall design, example models for each task, and performance benchmarking behind ESPnet-ST-v2, which is publicly available at https://github.com/espnet/espnet.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑