TAPO：多语言数学推理中的翻译增强策略优化
TAPO: Translation Augmented Policy Optimization for Multilingual Mathematical Reasoning
AI总结：
TAPO通过引入翻译增强策略优化框架，提升多语言数学推理能力，有效结合语言理解和推理能力，优于基线方法并在未见语言和领域任务中表现良好。
摘要：
大型语言模型（LLMs）在英语数学推理方面表现出色，但在多语言环境下仍存在显著性能差距，主要归因于语言理解不足。为弥合这一差距，我们引入了基于GRPO的翻译增强策略优化（TAPO）框架。TAPO采用显式对齐策略，以英语为桥梁，遵循先理解后推理的范式。关键在于采用步级相对优势机制，将理解与推理解耦，使翻译质量奖励的整合无需引入优化冲突。大量实验表明，TAPO有效融合了语言理解和推理能力，并兼容多种模型。它在多语言数学推理和翻译任务中均优于基线方法，且在未见语言和领域任务中表现良好。
英文摘要：
Large Language Models (LLMs) have demonstrated remarkable proficiency in English mathematical reasoning, yet a significant performance disparity persists in multilingual contexts, largely attributed to deficiencies in language understanding. To bridge this gap, we introduce Translation-Augmented Policy Optimization (TAPO), a novel reinforcement learning framework built upon GRPO. TAPO enforces an explicit alignment strategy where the model leverages English as a pivot and follows an understand-then-reason paradigm. Crucially, we employ a step-level relative advantage mechanism that decouples understanding from reasoning, allowing the integration of translation quality rewards without introducing optimization conflicts. Extensive experiments reveal that TAPO effectively synergizes language understanding with reasoning capabilities and is compatible with various models. It outperforms baseline methods in both multilingual mathematical reasoning and translation tasks, while generalizing well to unseen languages and out-of-domain tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
