TVPara：复杂声学环境下情感电视剧语音的副语言字幕生成
TVPara: Paralinguistic Captioning of Expressive TV-Series Speech in Complex Acoustic Environments
AI总结：
本研究构建了基于电视剧的副语言字幕数据集TVPara，通过微调开源大型音频-语言模型，显著提升了复杂声学环境下的副语言理解能力，且增益可泛化至未见过的电视剧及跨域语音。
摘要：
大型音频-语言模型（LALMs）难以解析结合强烈情感与复杂声学环境的语音中的副语言特征（如情感、语速、音调），提升该能力需要反映此类条件的训练数据，但现有语料库严重依赖录音室录制的语音，情感与声学多样性有限。为解决这一差距，我们推出TVPara——一个基于电视剧构建的副语言字幕数据集，电视剧语音常带有多样情感表达并伴随场景特定背景音，可大规模提供丰富情感与复杂声学环境。TVPara通过LALM预标注与人工校正构建，包含两个任务：单utterance（约492小时）和多utterance（约213小时）副语言字幕生成。在TVPara上微调开源LALMs，可在复杂声学环境下的副语言理解中平均相对提升31.26%，该增益还可泛化至未见过的电视剧及更清晰的跨域语音。
英文摘要：
Large audio-language models (LALMs) struggle to parse paralinguistics (e.g., emotion, pace, pitch) in speech that combines intense emotion with complex acoustic environments. Strengthening this ability calls for training data that reflects such conditions. However, existing corpora rely heavily on studio-recorded speech, offering limited emotional and acoustic diversity. To address this gap, we introduce TVPara, a paralinguistic captioning dataset built from TV series. Speech in TV series often carries diverse emotional expression and co-occurs with scene-specific background sound, providing expressive emotion and complex acoustic environments at scale. Constructed via LALM pre-annotation and human correction, TVPara includes two tasks: Single-Utterance (~492h) and Multi-Utterance (~213h) Paralinguistic Captioning. Fine-tuning open-source LALMs on TVPara yields an average relative improvement of 31.26% in paralinguistic understanding under complex acoustic environments. The gains further generalize to an unseen TV series and to cleaner, cross-domain speech.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
