探索跨语言潜在移植：相互机会与开放挑战
Exploring Cross-lingual Latent Transplantation: Mutual Opportunities and Open Challenges
AI总结：
本文提出跨语言潜在移植框架，探讨其对多语言能力和文化适应性的双向影响，揭示注意力模块与前馈模块在多语言理解中的作用，发现当前LLM多语言潜力的严重未利用问题。
摘要：
当前大型语言模型（LLMs）在多语言能力和文化适应性上常存在不平衡，主要归因于其以英语为中心的预训练数据。本文引入并研究了跨语言潜在移植（XTransplant），一种探测框架，旨在进一步挖掘模型内部化的多语言知识并在推理过程中检验其对LLMs多语言能力和文化适应性的影响。XTransplant框架通过在不同语言间移植潜在激活，使模型能够利用英语和非英语资源的互补优势。通过广泛分析，我们实证证明XTransplant作为一种跨语言交互形式，对LLMs的多语言能力和文化适应性具有相互促进的效果，尤其对低资源语言和文化。我们进一步揭示注意力模块在支持多语言理解中的关键作用，而前馈模块更擅长捕捉文化特定知识。此外，我们深入分析了XTransplant的稳定性、有效性和泛化能力。通过探测XTransplant的上限性能，我们揭示了当前LLM多语言潜力的严重未利用问题——一个仍需解决的挑战。我们希望本分析能为推进跨语言交互和更好地利用模型内部化的多语言知识提供新的视角。
英文摘要：
Current large language models (LLMs) often exhibit imbalances in multilingual capabilities and cultural adaptability, largely attributed to their English-centric pre-training data. In this paper, we introduce and investigate cross-lingual latent transplantation (XTransplant), a probing framework which aims to further exploit the model's internalized multilingual knowledge during inference and examine its effects on the multilingual capability and cultural adaptability of LLMs. XTransplant framework enables models to harness the complementary strengths of both English and non-English resources by transplanting latent activations across languages. Through extensive analysis, we empirically demonstrate that XTransplant, a form of cross-lingual interaction, has mutually beneficial effects on the multilingual capability and cultural adaptability of LLMs, particularly for low-resource languages and cultures. We further reveal that attention modules play a pivotal role in supporting multilingual understanding, while feed-forward modules are more adept at capturing culture-specific knowledge. In addition, we conduct in-depth analysis of XTransplant's stability, effectiveness, and generalizability. By probing the upper bound performance of XTransplant, we expose the considerable underutilization of current LLMs' multilingual potential-a challenge that remains open. We hope our analysis offers a new lens for advancing cross-lingual interactions and better leveraging models' internalized multilingual knowledge.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
