arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.17874cs.CLcs.AI
收藏

利用LLM生成的上下文描述改进特定领域的ASR

Improving Domain-Specific ASR with LLM-Generated Contextual Descriptions

Hanyang University(汉阳大学)

Jiwon Suh, Injae Na, Woohwan Jung

AI总结:

针对端到端ASR在特定领域词汇识别上的不足,提出在不修改Whisper架构的前提下利用上下文描述,并结合解码器微调与上下文扰动技术,同时引入LLM生成描述,显著提升了真实数据集上的识别准确率。

摘要:

端到端自动语音识别(E2E ASR)系统通过在海量数据集上训练,显著提升了语音识别性能。尽管取得这些进展,它们仍难以准确识别特定领域的词汇,如专有名词和技术术语。为解决此问题,我们提出一种利用最先进Whisper模型的方法,在不修改其架构的前提下,保持其泛化性能并使其能有效利用描述。此外,我们提出两种额外训练技术以改进特定领域ASR:解码器微调和上下文扰动。当描述不可用时,我们还提出一种使用大语言模型(LLM)结合简单元数据生成描述的方法。实验表明,所提方法在真实数据集上显著提升了特定领域ASR准确率,且LLM生成的描述在有效性上优于人工撰写的描述。

英文摘要:

End-to-end automatic speech recognition (E2E ASR) systems have significantly improved speech recognition through training on extensive datasets. Despite these advancements, they still struggle to accurately recognize domain specific words, such as proper nouns and technical terminologies. To address this problem, we propose a method to utilize the state-of-the-art Whisper without modifying its architecture, preserving its generalization performance while enabling it to leverage descriptions effectively. Moreover, we propose two additional training techniques to improve the domain specific ASR: decoder fine-tuning, and context perturbation. We also propose a method to use a Large Language Model (LLM) to generate descriptions with simple metadata, when descriptions are unavailable. Our experiments demonstrate that proposed methods notably enhance domain-specific ASR accuracy on real-life datasets, with LLM-generated descriptions outperforming human-crafted ones in effectiveness.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑