赋能聋人与听障群体:利用大语言模型增强视频字幕
Empowering the Deaf and Hard of Hearing Community: Enhancing Video Captions Using Large Language Models
AI总结:
针对ASR生成的视频字幕准确率低、难以满足聋人与听障群体需求的问题,本研究提出基于GPT-3.5、Llama2-13B等大语言模型的字幕校正流水线,在自建真实场景数据集上实现了约57.72%的词错误率下降。
摘要:
在当今数字时代,视频内容十分普及,是信息、教育与娱乐的主要来源。然而,由于自动语音识别(ASR)系统难以提供准确可靠的字幕,聋人与听障(DHH)群体在获取视频内容时往往面临重大挑战。本文针对提升视频字幕质量的迫切需求,提出利用大语言模型(LLM)解决该问题。我们开展了一项综合研究,探索通过整合LLM来提升ASR系统生成字幕的准确性与语境感知能力。我们的方法包含一套新颖的流水线,使用先进的LLM校正ASR生成的字幕,且因GPT-3.5和Llama2-13B在语言理解与生成任务中表现稳健,我们明确聚焦于这类模型。我们引入了一个能代表DHH群体所面临真实挑战的数据集,用于评估所提出的流水线。结果表明,经LLM增强的字幕准确性显著提升:ChatGPT-3.5的词错误率(WER)仅为9.75%,远低于原始ASR字幕的23.07%,相比原始ASR字幕,ChatGPT-3.5的WER约降低了57.72%。
英文摘要:
In today's digital age, video content is prevalent, serving as a primary source of information, education, and entertainment. However, the Deaf and Hard of Hearing (DHH) community often faces significant challenges in accessing video content due to the inadequacy of automatic speech recognition (ASR) systems in providing accurate and reliable captions. This paper addresses the urgent need to improve video caption quality by leveraging Large Language Models (LLMs). We present a comprehensive study that explores the integration of LLMs to enhance the accuracy and context-awareness of captions generated by ASR systems. Our methodology involves a novel pipeline that corrects ASR-generated captions using advanced LLMs. It explicitly focuses on models like GPT-3.5 and Llama2-13B due to their robust performance in language comprehension and generation tasks. We introduce a dataset representative of real-world challenges the DHH community faces to evaluate our proposed pipeline. Our results indicate that LLM-enhanced captions significantly improve accuracy, as evidenced by a notably lower Word Error Rate (WER) achieved by ChatGPT-3.5 (WER: 9.75%) compared to the original ASR captions (WER: 23.07%), ChatGPT-3.5 shows an approximate 57.72% improvement in WER compared to the original ASR captions.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。