多语言链式思维推理的综合评估：跨语言的性能、一致性与忠实性
A Comprehensive Evaluation of Multilingual Chain-of-Thought Reasoning: Performance, Consistency, and Faithfulness Across Languages
AI总结：
本文首次全面评估多语言链式思维推理，从性能、一致性和忠实性三个维度分析，发现语言偏好显著影响思维轨迹质量，并发布代码与数据。
摘要：
大型推理模型（LRMs）越来越依赖逐步的链式思维（CoT）推理来提高任务性能，尤其是在英语等高资源语言中。尽管近期研究已考察了多语言环境下的最终答案准确性，但思维轨迹本身，即通向最终答案的中间步骤，仍未被充分探索。在本文中，我们首次对多语言CoT推理进行了全面研究，评估三个关键维度：性能、一致性和忠实性。我们首先测量当LRMs被明确指示或通过提示攻击（prompt-hacked）以目标语言思考时的语言遵从性、答案准确性和答案一致性，揭示了强烈的语言偏好和跨语言的性能差异。接下来，我们通过在不同语言之间互换思维轨迹来评估其跨语言一致性。我们发现思维轨迹的质量和有效性因提示语言的不同而有显著差异。最后，我们采用基于扰动的技术——即截断和错误注入——来探测跨语言思维轨迹的忠实性，表明模型对轨迹的依赖程度各不相同。我们发布代码和数据以支持未来研究。
英文摘要：
Large reasoning models (LRMs) increasingly rely on step-by-step Chain-of-Thought (CoT) reasoning to improve task performance, particularly in high-resource languages such as English. While recent work has examined final-answer accuracy in multilingual settings, the thinking traces themselves, i.e., the intermediate steps that lead to the final answer, remain underexplored. In this paper, we present the first comprehensive study of multilingual CoT reasoning, evaluating three key dimensions: performance, consistency, and faithfulness. We begin by measuring language compliance, answer accuracy, and answer consistency when LRMs are explicitly instructed or prompt-hacked to think in a target language, revealing strong language preferences and divergent performance across languages. Next, we assess crosslingual consistency of thinking traces by interchanging them between languages. We find that the quality and effectiveness of thinking traces vary substantially depending on the prompt language. Finally, we adapt perturbation-based techniques -- i.e., truncation and error injection -- to probe the faithfulness of thinking traces across languages, showing that models rely on traces to varying degrees. We release our code and data to support future research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
