arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.03093cs.CLcs.SD

重新审视基于语音大语言模型的直接语音到文本翻译:比CoT提示更好的扩展性?

Revisiting Direct Speech-to-Text Translation with Speech LLMs: Better Scaling than CoT Prompting?

Barcelona Supercomputing Center, Spain(巴塞罗那超级计算中心,西班牙);Universitat Politècnica de Catalunya, Spain(加泰罗尼亚理工大学,西班牙);DFKI GmbH, Saarland Informatics Campus, Saarbrücken, Germany(德意志联邦计算机研究中心(DFKI) GmbH,萨尔布吕肯,德国)

Oriol Pareras, Gerard I. Gállego, Federico Costa, Cristina España-Bonet, Javier Hernando

AI总结:

本文比较了CoT和直接提示在增加S2TT数据量下的表现,发现直接提示在数据增加时表现更优,可能成为更大资源下的更有效方法。

摘要:

近期语音到文本翻译(S2TT)研究聚焦于基于LLM的模型,引入了日益采用的链式思维(CoT)提示,即模型首先转录语音然后翻译。CoT通常优于直接提示,主要是因为它可以利用大量自动语音识别(ASR)和文本到文本翻译(T2TT)数据集来显式建模其步骤。在本文中,我们系统比较了CoT和直接提示在增加S2TT数据量下的表现。为此,我们通过将ASR语料库的转录翻译成六种欧洲语言进行伪标签,并在不同数据规模下训练基于LLM的S2TT系统。我们的结果表明,随着数据量的增加,直接提示表现更加一致,表明它可能在更大S2TT资源下成为更有效的方法。

英文摘要:

Recent work on Speech-to-Text Translation (S2TT) has focused on LLM-based models, introducing the increasingly adopted Chain-of-Thought (CoT) prompting, where the model is guided to first transcribe the speech and then translate it. CoT typically outperforms direct prompting primarily because it can exploit abundant Automatic Speech Recognition (ASR) and Text-to-Text Translation (T2TT) datasets to explicitly model its steps. In this paper, we systematically compare CoT and Direct prompting under increasing amounts of S2TT data. To this end, we pseudo-label an ASR corpus by translating its transcriptions into six European languages, and train LLM-based S2TT systems with both prompting strategies at different data scales. Our results show that Direct improves more consistently as the amount of data increases, suggesting that it may become a more effective approach as larger S2TT resources are created.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑