arXiv 2505.199142025-06-09cs.CLcs.AI

Enigmata：利用合成可验证谜题扩展大型语言模型的逻辑推理能力

Enigmata: Scaling Logical Reasoning in Large Language Models with Synthetic Verifiable Puzzles

ByteDance Seed（字节跳动种子）；Fudan University（复旦大学）；Institute for AI Industry Research (AIR), Tsinghua University（人工智能产业研究院（AIR），清华大学）；Nanjing University（南京大学）；Shanghai Jiao Tong University（上海交通大学）；SIA-Lab of Tsinghua AIR and ByteDance Seed（清华大学AIR实验室和字节跳动种子）

Jiangjie Chen, Qianyu He, Siyu Yuan, Aili Chen, Zhicheng Cai, Weinan Dai, Hongli Yu, Qiying Yu, Xuefeng Li, Jiaze Chen, Hao Zhou, Mingxuan Wang

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出含36类可生成、可验证谜题的 Enigmata 框架与 Enigmata-Eval，并通过多任务 RLVR 提升 Qwen2.5-32B 及 Seed1.5-Thinking 的谜题、数学和 STEM 推理性能。

摘要：

大型语言模型（LLM），例如 OpenAI 的 o1 和 DeepSeek 的 R1，通过带可验证奖励的强化学习（RLVR）在数学和编程等高级推理任务上表现出色，但在处理人类无需领域知识即可求解的谜题时仍有困难。我们提出 Enigmata，这是首个专为提升 LLM 谜题推理能力而设计的综合套件。它包含横跨七大类别的 36 项任务，每项任务都具备：1）一个可生成无限数量且难度可控示例的生成器；2）一个用于自动评估的基于规则的验证器。这种生成器—验证器设计支持可扩展的多任务强化学习训练、细粒度分析以及与 RLVR 的无缝集成。我们进一步提出严格的基准 Enigmata-Eval，并开发了优化的多任务 RLVR 策略。我们训练得到的模型 Qwen2.5-32B-Enigmata 在 Enigmata-Eval、ARC-AGI（32.8%）和 ARC-AGI 2（0.6%）等谜题推理基准上持续超越 o3-mini-high 与 o1。它还能很好地泛化到域外谜题基准和数学推理，并且几乎没有多任务权衡。当在 Seed1.5-Thinking（激活参数 20B、总参数 200B）等更大模型上训练时，来自 Enigmata 的谜题数据进一步提升了其在 AIME（2024–2025）、BeyondAIME 和 GPQA（Diamond）等高级数学与 STEM 推理任务上的 SoTA 性能，显示出 Enigmata 良好的泛化收益。本工作为推进 LLM 的逻辑推理提供了统一、可控的框架。本工作资源可在 https://seed-enigmata.github.io 获取。

英文摘要：

Large Language Models (LLMs), such as OpenAI's o1 and DeepSeek's R1, excel at advanced reasoning tasks like math and coding via Reinforcement Learning with Verifiable Rewards (RLVR), but still struggle with puzzles solvable by humans without domain knowledge. We introduce Enigmata, the first comprehensive suite tailored for improving LLMs with puzzle reasoning skills. It includes 36 tasks across seven categories, each with 1) a generator that produces unlimited examples with controllable difficulty and 2) a rule-based verifier for automatic evaluation. This generator-verifier design supports scalable, multi-task RL training, fine-grained analysis, and seamless RLVR integration. We further propose Enigmata-Eval, a rigorous benchmark, and develop optimized multi-task RLVR strategies. Our trained model, Qwen2.5-32B-Enigmata, consistently surpasses o3-mini-high and o1 on the puzzle reasoning benchmarks like Enigmata-Eval, ARC-AGI (32.8%), and ARC-AGI 2 (0.6%). It also generalizes well to out-of-domain puzzle benchmarks and mathematical reasoning, with little multi-tasking trade-off. When trained on larger models like Seed1.5-Thinking (20B activated parameters and 200B total parameters), puzzle data from Enigmata further boosts SoTA performance on advanced math and STEM reasoning tasks such as AIME (2024-2025), BeyondAIME and GPQA (Diamond), showing nice generalization benefits of Enigmata. This work offers a unified, controllable framework for advancing logical reasoning in LLMs. Resources of this work can be found at https://seed-enigmata.github.io.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。