arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.09489cs.SDcs.MMeess.AS

ecVoice:基于习语相似度替换的视频音频文本提取与优化

ecVoice: Audio Text Extraction and Optimization of Video Based on Idioms Similarity Replacement

Monash University(莫纳什大学)

Jinwei Lin

AI总结:

针对低计算内存下Whisper语音识别受限的问题,提出基于习语相似度计算分析的ecVoice方法,可提升音频文本提取质量,将习语语法纠错率平均提至90%,低内存下增强Whisper识别效果,可用于视频翻译等场景。

摘要:

视频中的音频文本提取在多媒体编辑与处理中发挥着重要作用。作为一款广受欢迎的开源工具包,Whisper在人声识别方面运行速度快。然而,其识别性能依赖于计算资源,这使得低计算内存环境下运行Whisper变得困难。本文提出了一种可行方案,用于从视频中提取人声并从语音中生成高质量文本,生成的语音可应用于视频语言翻译和译制语音模拟。为提升人声提取与转换质量,我们提出了ecVoice方法,该方法利用习语相似度计算与分析来提高音频文本提取的质量。我们开展了相关实验,验证了ecVoice可将习语语法纠错率平均提升至90%。该方法简单且快速,意味着在提高语音识别率的同时,它对计算资源消耗造成的负面影响更小。我们的方法与方案能够在低计算内存条件下显著增强Whisper的识别效果。

英文摘要:

The Text Extraction of the Audio from the Video plays an important role in multimedia editing and processing. As a popular open source toolkit, Whisper performs fast in human voice recognition. However, the recognition performance is dependent on the computing resource, which makes the low computing memory running Whisper become difficult. Our paper presents an available solution to extract the human voice from the video and gain the high quality text generation from the voice. The generated voice can be used in video language translation and translated voice simulation. To improve the extraction and transform quality of human voice, we present ecVoice, a method using the idioms similarity computation and analysis to improve the quality of audio text extraction. Relative experiments are held to verify that the ecVoice can improve the idiom grammar correction rate to 90\% on average. The method is simple but fast which means this method will cause less bad influence of consuming computing resources when improving the voice recognition rate. Our method and solution can significantly enhance the Whisper recognition with low computing memory.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑