arXiv 2405.183592024-05-28cs.CLcs.AIcs.LG

弥合差距：提升大型语言模型多语言性能的动态学习策略

Bridging the Gap: Dynamic Learning Strategies for Improving Multilingual Performance in LLMs

Microsoft Research India（微软研究院印度分院）

Somnath Kumar, Vaibhav Balloli, Mercy Ranjit, Kabir Ahuja, Tanuja Ganu, Sunayana Sitaram, Kalika Bali, Akshay Nambi

备注：: arXiv admin note: text overlap with arXiv:2305.17740

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文针对 LLM 在非拉丁文字和低资源语言上的性能不足，提出提示优化、RAG 与多语言嵌入混合及运行时动态选择策略，显著提升多语言任务表现。

摘要：

大型语言模型（LLM）正处于全球众多领域变革的前沿。然而，对于非拉丁文字和低资源语言，其包容性和有效性仍然有限。本文应对在不进行大规模训练或微调的情况下提升 LLM 多语言性能这一迫切挑战。通过使用流行的问答（QA）数据集对多种语言进行系统调查和评估，我们提出了新技术，以释放 LLM 在多语言环境中的真正潜力。我们的方法包含三项关键策略，可显著提升多语言能力。首先，通过精心优化为多语言 LLM 量身定制的提示词，我们释放了其潜在能力，从而带来跨语言性能的大幅提升。其次，我们提出一种新的混合方法，将 LLM 检索增强生成（RAG）与多语言嵌入协同结合，实现了更好的多语言任务性能。最后，我们提出一种新颖的学习方法，在运行时针对每个查询动态选择最优提示策略、LLM 模型和嵌入模型。这种动态适应最大化了 LLM 跨语言的效能，优于最佳静态策略和随机策略。此外，我们的方法可在离线和在线环境中调整配置，并能无缝适应新语言和数据集，从而推动多种语言在多语言理解与生成方面的实质性进步。

英文摘要：

Large language models (LLMs) are at the forefront of transforming numerous domains globally. However, their inclusivity and effectiveness remain limited for non-Latin scripts and low-resource languages. This paper tackles the imperative challenge of enhancing the multilingual performance of LLMs without extensive training or fine-tuning. Through systematic investigation and evaluation of diverse languages using popular question-answering (QA) datasets, we present novel techniques that unlock the true potential of LLMs in a polyglot landscape. Our approach encompasses three key strategies that yield significant improvements in multilingual proficiency. First, by meticulously optimizing prompts tailored for polyglot LLMs, we unlock their latent capabilities, resulting in substantial performance boosts across languages. Second, we introduce a new hybrid approach that synergizes LLM Retrieval Augmented Generation (RAG) with multilingual embeddings and achieves improved multilingual task performance. Finally, we introduce a novel learning approach that dynamically selects the optimal prompt strategy, LLM model, and embedding model per query at run-time. This dynamic adaptation maximizes the efficacy of LLMs across languages, outperforming best static and random strategies. Additionally, our approach adapts configurations in both offline and online settings, and can seamlessly adapt to new languages and datasets, leading to substantial advancements in multilingual understanding and generation across diverse languages.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。