字典插入提示用于多语言推理在多语言大语言模型上
Dictionary Insertion Prompting for Multilingual Reasoning on Multilingual Large Language Models
AI总结：
本文提出了一种名为字典插入提示（DIP）的新方法，通过在提示中插入词典中的英文对应词来提升多语言推理性能，实验表明在10到200种语言上效果显著。
摘要：
在当前的大语言模型（LLMs）时代，存在两个不足：一是缺乏多语言模型，大多数LLMs以英语为中心，多语言推理性能受限；二是外部知识的使用位置，大多数检索的知识被前置到用户查询中（可能不最优）。本文提出了一种新颖且有效的称为字典插入提示（DIP）的方法。当提供非英语提示时，DIP会查找词典并插入词的英文对应词到提示的中间部分，从而使LLMs更好地翻译成英语并产生更好的英语模型思考步骤，从而获得明显更好的结果。我们实验了10到200种语言（FLORES-200）。由于没有足够的数据集，我们使用NLLB翻译器从现有的4个英语推理基准（如GSM8K和AQuA）创建合成的多语言基准。合成基准被翻译回英语以确保质量并通过人工标注。有趣的是，插入词典的位置对性能提升有重要影响，我们发现在原始词和词典之间交替插入比前置或后置词典效果更好，同一词典构建下。
英文摘要：
There are two shortages in the current Large Language Models (LLMs) era. The first is short of multilingual models, where most LLMs are English-centric and performance is limited on multilingual reasoning. The second is the place of external knowledge to be used, where most retrieved knowledge is prepended to the user queries (maybe sub-optimal). This paper presents a novel and simple yet effective method called \textbf{D}ictionary \textbf{I}nsertion \textbf{P}rompting (\textbf{DIP}). When providing a non-English prompt, DIP looks up a word dictionary and inserts words' English counterparts into the middle of the prompt for LLMs. It then enables better translation into English and better English model thinking steps which leads to obviously better results. We experiment with 10 to 200 languages from FLORES-200.\footnote{The number of languages varies on the datasets, and we experiment with 200 languages on GSM8K as in Appendix} Since there are no adequate datasets, we use the NLLB translator to create synthetic multilingual benchmarks from the existing 4 English reasoning benchmarks such as GSM8K and AQuA. The synthetic benchmarks are translated back into English for quality assurance with manual annotation. Interestingly, the place for injecting the dictionary plays an important factor in the performance gains, and we found that interleaving the dictionary with the original words gives a better performance compared to prepending/appending the dictionary, under the same dictionary constructed.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
