arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.00889cs.CLcs.AI
收藏

CHIMERA:紧凑的合成数据用于通用的LLM推理

CHIMERA: Compact Synthetic Data for Generalizable LLM Reasoning

University of Virginia(弗吉尼亚大学);Apple(苹果公司);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校)

Xinyu Zhu, Yihao Feng, Yanchao Sun, Xianzhi Du, Pingzhi Li, Olli Saarikivi, Yun Zhu, Yu Meng

AI总结:

CHIMERA通过紧凑的合成数据集提升LLM跨领域推理能力,提供丰富推理轨迹和结构化覆盖,实现高效训练和评估。

摘要:

大型语言模型(LLMs)最近展现出显著的推理能力,这主要得益于监督微调(SFT)和强化学习(RL)等基于高质量推理数据的后训练方法。然而,在开放和可扩展的环境中复制和扩展这些能力受到三个根本性的数据中心挑战的阻碍:(1)冷启动问题,源于缺乏具有详细、长链式思维(CoT)轨迹的种子数据集,以初始化推理策略;(2)领域覆盖有限,因为大多数现有的开源推理数据集集中在数学领域,对更广泛的科学学科覆盖有限;(3)标注瓶颈,前沿级别的推理任务难度使得可靠的标注成本高昂或不可行。为了解决这些挑战,我们引入了CHIMERA,一个包含9000个样本的紧凑合成推理数据集,用于通用的跨领域推理。CHIMERA具有三个关键特性:(1)它提供了由最先进的推理模型合成的丰富、长链式思维推理轨迹;(2)它具有广泛的、结构化的覆盖,涵盖8个主要科学学科和超过1000个细粒度主题,通过模型生成的层级分类法进行组织;(3)它采用完全自动化的、可扩展的评估流程,利用强大的推理模型交叉验证问题的正确性和答案的正确性。我们使用CHIMERA对400亿参数的Qwen3模型进行后训练。尽管数据集规模较小,但所得到的模型在一系列具有挑战性的推理基准测试中表现出色,包括GPQA-Diamond、AIME 24/25/26、HMMT 25和Humanity's Last Exam,接近或匹配大型模型如DeepSeek-R1和Qwen3-235B的推理性能。

英文摘要:

Large Language Models (LLMs) have exhibited remarkable reasoning capabilities, largely enabled by supervised fine-tuning (SFT)- and reinforcement learning (RL)-based post-training on high-quality reasoning data. However, reproducing and extending these capabilities in open and scalable settings is hindered by three data-centric challenges: (1) the cold-start problem, arising from the lack of seed datasets with detailed, long Chain-of-Thought (CoT) trajectories needed to initialize reasoning policies; (2) limited domain coverage, as most existing open-source reasoning datasets are concentrated in mathematics, with limited coverage of broader scientific disciplines; and (3) the annotation bottleneck, where the difficulty of frontier-level reasoning tasks makes reliable human annotation prohibitively expensive or infeasible. To address these challenges, we introduce CHIMERA, a compact synthetic reasoning dataset of 9K samples designed to support generalizable reasoning across domains. CHIMERA is constructed with three key properties: (1) it provides rich, long CoT reasoning trajectories synthesized by state-of-the-art reasoning models; (2) it has broad and structured coverage, spanning 8 major scientific disciplines and over 1K fine-grained topics organized via a model-generated hierarchical taxonomy; and (3) it employs a fully automated, scalable evaluation pipeline that uses strong reasoning models to cross-validate both problem validity and answer correctness, removing the reliance on human annotations. We use CHIMERA to post-train a 4B Qwen3 model with SFT and RL. Despite the dataset's modest size, the resulting model achieves strong performance on a suite of challenging reasoning benchmarks, including GPQA-Diamond, AIME 24/25/26, HMMT 25 and Humanity's Last Exam, approaching or matching the performance of substantially larger models such as DeepSeek-R1 and Qwen3-235B.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑