arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2505.21458cs.CL
收藏

大语言模型需要用一种语言思考吗?潜在语言与任务性能之间的相关性

Do LLMs Need to Think in One Language? Correlation between Latent Language and Task Performance

NAIST(日本国立信息与科技研究所);NII LLMC(日本信息处理学会大语言模型中心);MBZUAI(微软亚洲人工智能研究院);RIKEN(日本研究机构);Tohoku University(东北大学);The University of Tokyo(东京大学);Nagoya Institute of Technology(名古屋技术大学)

Shintaro Ozaki, Tatsuya Hiraoka, Hiroto Otake, Hiroki Ouchi, Masaru Isonuma, Benjamin Heinzerling, Kentaro Inui, Taro Watanabe, Yusuke Miyao, Yohei Oseki, Yu Takagi

AI总结:

本研究探讨了LLMs潜在语言与任务性能的相关性,发现因模型在最终层调整内部表示以匹配目标语言,保持潜在语言一致性并非总是提升下游任务性能的必要条件。

摘要:

大型语言模型(LLMs)已知会持续使用一种熟练的内部语言来处理信息,这被称为潜在语言,它可能与输入或输出语言不同。然而,潜在语言与输入和输出语言之间的差异如何影响下游任务性能,在很大程度上仍未被探索。虽然许多研究探讨了LLMs的潜在语言,但很少有研究关注其在影响任务性能方面的重要性。在我们的研究中,我们假设用潜在语言一致地思考能提升下游任务性能。为了验证这一点,我们的工作在多个下游任务中改变了输入提示语言,并分析了潜在语言一致性与任务性能之间的相关性。我们创建了包含来自翻译和地理文化等不同领域问题的数据集,这些问题受潜在语言选择的影响。在多个LLMs上针对翻译和地理文化任务(这些任务对语言选择敏感)的实验结果表明,保持潜在语言的一致性对于获得最佳的下游任务性能并不总是必要的。这是因为这些模型会在接近最终层时调整其内部表示以匹配目标语言,从而降低一致性对整体性能的影响。

英文摘要:

Large Language Models (LLMs) are known to process information using a proficient internal language consistently, referred to as latent language, which may differ from the input or output languages. However, how the discrepancy between the latent language and the input and output language affects downstream task performance remains largely unexplored. While many studies research the latent language of LLMs, few address its importance in influencing task performance. In our study, we hypothesize that thinking in latent language consistently enhances downstream task performance. To validate this, our work varies the input prompt languages across multiple downstream tasks and analyzes the correlation between consistency in latent language and task performance. We create datasets consisting of questions from diverse domains such as translation and geo-culture, which are influenced by the choice of latent language. Experimental results across multiple LLMs on translation and geo-culture tasks, which are sensitive to the choice of language, indicate that maintaining consistency in latent language is not always necessary for optimal downstream task performance. This is because these models adapt their internal representations near the final layers to match the target language, reducing the impact of consistency on overall performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑