arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2209.13192cs.CL
收藏

面向自动字幕生成的直接语音翻译

Direct Speech Translation for Automatic Subtitling

Fondazione Bruno Kessler(布鲁诺·凯斯勒基金会);University of Trento(特伦托大学);Leiden University Centre for Linguistics(莱顿大学语言学中心);Zoom Video Communications(Zoom视频通讯)

Sara Papi, Marco Gaido, Alina Karakanta, Mauro Cettolo, Matteo Negri, Marco Turchi

AI总结:

本文提出首个用于自动字幕生成的直接ST模型,利用单一模型直接生成目标语言字幕及时间戳,在7个语言对实验中优于级联系统且与生产工具竞争。

摘要:

自动字幕生成是指将视听内容中的语音自动翻译为带时间戳的短文本片段(即字幕及其对应时间戳)的任务。生成的字幕需满足空间和时间限制,同时与语音同步,并以易于理解的方式进行分段。鉴于任务复杂性,目前通常通过级联组件分别处理语音转写、翻译、文本分段为字幕以及预测时间戳。本文提出首个用于自动字幕生成的直接ST模型,该模型利用单一模型直接生成目标语言字幕及其对应的时间戳。我们在7个语言对上的实验表明,在相同数据条件下,该方法优于级联系统,并且在覆盖新场景的领域内和新发布的领域外基准上,与现有生产工具相比也具有竞争力。

英文摘要:

Automatic subtitling is the task of automatically translating the speech of audiovisual content into short pieces of timed text, i.e. subtitles and their corresponding timestamps. The generated subtitles need to conform to space and time requirements, while being synchronised with the speech and segmented in a way that facilitates comprehension. Given its considerable complexity, the task has so far been addressed through a pipeline of components that separately deal with transcribing, translating, and segmenting text into subtitles, as well as predicting timestamps. In this paper, we propose the first direct ST model for automatic subtitling that generates subtitles in the target language along with their timestamps with a single model. Our experiments on 7 language pairs show that our approach outperforms a cascade system in the same data condition, also being competitive with production tools on both in-domain and newly-released out-domain benchmarks covering new scenarios.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑