利用广播媒体字幕转录文本提升自动语音识别与字幕生成
Leveraging Broadcast Media Subtitle Transcripts for Automatic Speech Recognition and Subtitling
AI总结：
本文研究将电视字幕弱监督转录与逐字转录作为不同领域联合建模，提出级联编码器和分离解码器的端到端ASR架构，在弗拉芒语上同时提升语音识别准确率和字幕质量。
摘要：
语音识别技术的最新进展得益于大规模数据集和基于注意力的架构，但许多挑战仍然存在，尤其是对于低资源语言和方言。本文探讨了将来自电视字幕的弱监督转录文本整合到自动语音识别（ASR）系统中，旨在同时改进逐字转录和自动生成的字幕。为此，由于逐字数据和字幕具有不同的特征，它们被视为不同的领域或语言。我们提出并比较了几种端到端架构，这些架构旨在通过分离或共享的编码器和解码器来联合建模两种模态。所提出的方法能够联合生成逐字转录和字幕。在弗拉芒语（比利时荷兰语）上的评估表明，具有级联编码器和分离解码器的模型能够最有效地表示两种数据类型之间的差异，同时在两个领域上都有所改进。尽管在领域和语言变异方面存在差异，将逐字转录与字幕数据相结合可以显著改善ASR，而无需大量预处理。此外，使用大规模字幕数据集进行的实验显示了所提出方法的可扩展性。这些方法不仅提高了ASR准确率，还能生成与标准书面文本高度匹配的字幕，从而提供了多种潜在应用。
英文摘要：
The recent advancement of speech recognition technology has been driven by large-scale datasets and attention-based architectures, but many challenges still remain, especially for low-resource languages and dialects. This paper explores the integration of weakly supervised transcripts from TV subtitles into automatic speech recognition (ASR) systems, aiming to improve both verbatim transcriptions and automatically generated subtitles. To this end, verbatim data and subtitles are regarded as different domains or languages, due to their distinct characteristics. We propose and compare several end-to-end architectures that are designed to jointly model both modalities with separate or shared encoders and decoders. The proposed methods are able to jointly generate a verbatim transcription and a subtitle. Evaluation on Flemish (Belgian Dutch) demonstrates that a model with cascaded encoders and separate decoders allows to represent the differences between the two data types most efficiently while improving on both domains. Despite differences in domain and linguistic variations, combining verbatim transcripts with subtitle data leads to notable ASR improvements without the need for extensive preprocessing. Additionally, experiments with a large-scale subtitle dataset show the scalability of the proposed approach. The methods not only improve ASR accuracy but also generate subtitles that closely match standard written text, offering several potential applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
