SynLogic：大规模合成可验证推理数据以学习逻辑推理及其更广泛能力
SynLogic: Synthesizing Verifiable Reasoning Data at Scale for Learning Logical Reasoning and Beyond
AI总结：
SynLogic提出可大规模合成、难度可调且规则可验证的逻辑推理数据框架与数据集，并通过RL训练显著提升LLMs逻辑推理及跨领域泛化能力。
摘要：
近期如OpenAI-o1和DeepSeek R1等进展已经展示了强化学习（RL）在增强大语言模型（LLMs）推理能力方面的潜力。尽管开源复现工作主要聚焦于数学和编程领域，但用于发展通用推理能力的方法和资源仍未被充分探索。这一空白部分源于收集适合RL的多样化且可验证推理数据的挑战。我们假设逻辑推理对于发展通用推理能力至关重要，因为逻辑构成了推理的基本构件。在本工作中，我们提出了SynLogic，一个数据合成框架和数据集，能够大规模生成多样化的逻辑推理数据，涵盖35个多样化的逻辑推理任务。SynLogic方法能够以可调节的难度和数量进行受控数据合成。重要的是，所有示例都可以通过简单规则进行验证，使其非常适合具有可验证奖励的RL。在我们的实验中，我们基于7B和32B模型验证了在SynLogic数据集上进行RL训练的有效性。SynLogic在开源数据集中取得了最先进的逻辑推理性能，在BBEH上超过DeepSeek-R1-Distill-Qwen-32B 6个点。此外，将SynLogic数据与数学和编程任务混合可提高这些领域的训练效率，并显著增强推理泛化能力。值得注意的是，我们的混合训练模型在多个基准上优于DeepSeek-R1-Zero-Qwen-32B。这些发现使SynLogic成为推进LLMs更广泛推理能力的宝贵资源。我们在https://github.com/MiniMax-AI/SynLogic开源了数据合成流程和SynLogic数据集。
英文摘要：
Recent advances such as OpenAI-o1 and DeepSeek R1 have demonstrated the potential of Reinforcement Learning (RL) to enhance reasoning abilities in Large Language Models (LLMs). While open-source replication efforts have primarily focused on mathematical and coding domains, methods and resources for developing general reasoning capabilities remain underexplored. This gap is partly due to the challenge of collecting diverse and verifiable reasoning data suitable for RL. We hypothesize that logical reasoning is critical for developing general reasoning capabilities, as logic forms a fundamental building block of reasoning. In this work, we present SynLogic, a data synthesis framework and dataset that generates diverse logical reasoning data at scale, encompassing 35 diverse logical reasoning tasks. The SynLogic approach enables controlled synthesis of data with adjustable difficulty and quantity. Importantly, all examples can be verified by simple rules, making them ideally suited for RL with verifiable rewards. In our experiments, we validate the effectiveness of RL training on the SynLogic dataset based on 7B and 32B models. SynLogic leads to state-of-the-art logical reasoning performance among open-source datasets, surpassing DeepSeek-R1-Distill-Qwen-32B by 6 points on BBEH. Furthermore, mixing SynLogic data with mathematical and coding tasks improves the training efficiency of these domains and significantly enhances reasoning generalization. Notably, our mixed training model outperforms DeepSeek-R1-Zero-Qwen-32B across multiple benchmarks. These findings position SynLogic as a valuable resource for advancing the broader reasoning capabilities of LLMs. We open-source both the data synthesis pipeline and the SynLogic dataset at https://github.com/MiniMax-AI/SynLogic.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
