通过翻译-推理联合训练实现自我改进的多语言长推理
R3S: Refining and Recovering Reinforcement Signals for Multilingual Understanding and Reasoning
AI总结：
TRIT通过整合翻译训练提升多语言长推理能力，在MMATH上超越基线7个百分点，提升答案正确性和语言一致性。
摘要：
长推理模型在多语言环境下往往表现不佳：它们倾向于用英语来回答非英语问题；当被限制在问题语言中进行推理时，准确性会大幅下降。这种困难源于多语言问题理解和多语言推理能力的限制。为了解决这两个问题，我们提出了TRIT（翻译-推理联合训练），这是一种自我改进的框架，将翻译训练整合到多语言推理中。在没有外部反馈或额外多语言数据的情况下，我们的方法联合提升了多语言问题理解和回答生成能力。在MMATH上，我们的方法平均比多个基线高出7个百分点，提高了答案正确性和语言一致性。进一步分析表明，整合翻译训练使跨语言问题对齐提高了超过10个百分点，并提升了数学问题和通用领域文本的翻译质量，FLORES-200上的增益高达8.4 COMET点。
英文摘要：
Large reasoning models often default to English reasoning when processing non-English questions, yet their performance drops substantially when reasoning in the question language. Even with the same reasoning language, semantically equivalent English and non-English questions still exhibit a clear performance gap. Together, these phenomena reveal two distinct bottlenecks: target-language question understanding and target-language reasoning. Existing methods typically optimize only one of these capabilities. However, simply combining them may not be sufficient to optimize both effectively, as answer correctness alone cannot distinguish failures in question understanding from those in reasoning. We propose R3S, a reinforcement learning framework that disentangles the optimization of the two capabilities. R3S refines translation rewards derived from downstream reasoning accuracy through English-solvability filtering and recovers target-language RLVR signals using self-generated English hints. Together, these designs require neither external model feedback nor external multilingual training data. Experiments across three backbone models and five languages show that R3S improves language-consistent accuracy over the target-language RLVR baseline on MMATH by an average of 10.3 percentage points, while maintaining near-perfect language consistency. Consistent gains on MMLU-ProX further demonstrate its generalization beyond math problems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
