arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2411.05945cs.CLcs.AIcs.LGcs.MAeess.AS
收藏

NeKo:跨模态识别后纠错与任务引导的专家混合语言模型

NeKo: Cross-Modality Post-Recognition Error Correction with Tasks-Guided Mixture-of-Experts Language Model

NVIDIA
Yen-Ting Lin, Zhehuai Chen, Piotr Zelasko, Zhen Wan, Xuesong Yang, Zih-Ching Chen, Krishna C Puvvada, Szu-Wei Fu, Ke Hu, Jun Wei Chiu, Jagadeesh Balam, Boris Gi… 展开作者

Yen-Ting Lin, Zhehuai Chen, Piotr Zelasko, Zhen Wan, Xuesong Yang, Zih-Ching Chen, Krishna C Puvvada, Szu-Wei Fu, Ke Hu, Jun Wei Chiu, Jagadeesh Balam, Boris Ginsburg, Yu-Chiang Frank Wang, Chao-Han Huck Yang

AI总结:

NeKo通过任务引导的专家混合模型,实现跨模态识别后的高效纠错,显著降低WER并提升BLEU分数。

摘要:

构建一个通用的后识别纠错器面临关键问题:如何在大量领域数据集的混合中最有效地训练模型?答案在于学习领域特定特征并在单个模型中消化其知识。以往的方法通过使用单独的纠错语言模型来实现这一点,导致参数显著增加。在本工作中,我们提出专家混合(MoE)作为解决方案,强调MoE远不止是一种可扩展性工具。我们提出了一种多任务纠错MoE,其中训练专家成为语音到文本、语言到文本和视觉到文本数据集的“专家”,通过学习将每个数据集的标记路由到其映射的专家。在Open ASR Leaderboard上的实验表明,我们通过实现平均相对5.0%的WER减少和语音和翻译任务中BLEU分数的显著提升,探索了新的最先进的性能。在零样本评估中,NeKo在Hyporadise基准测试中以15.5%到27.6%的相对WER减少优于GPT-3.5和Claude-Opus。NeKo在语法和后OCR纠错任务中作为多任务模型表现出竞争力。

英文摘要:

Construction of a general-purpose post-recognition error corrector poses a crucial question: how can we most effectively train a model on a large mixture of domain datasets? The answer would lie in learning dataset-specific features and digesting their knowledge in a single model. Previous methods achieve this by having separate correction language models, resulting in a significant increase in parameters. In this work, we present Mixture-of-Experts as a solution, highlighting that MoEs are much more than a scalability tool. We propose a Multi-Task Correction MoE, where we train the experts to become an ``expert'' of speech-to-text, language-to-text and vision-to-text datasets by learning to route each dataset's tokens to its mapped expert. Experiments on the Open ASR Leaderboard show that we explore a new state-of-the-art performance by achieving an average relative 5.0% WER reduction and substantial improvements in BLEU scores for speech and translation tasks. On zero-shot evaluation, NeKo outperforms GPT-3.5 and Claude-Opus with 15.5% to 27.6% relative WER reduction in the Hyporadise benchmark. NeKo performs competitively on grammar and post-OCR correction as a multi-task model.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑