MegaScience：推动科学推理后训练数据集的前沿发展
MegaScience: Pushing the Frontiers of Post-Training Datasets for Science Reasoning
AI总结：
本文提出TextbookReasoning和MegaScience两个大规模科学推理数据集，并构建全面评估系统，实验证明其能显著提升多个开源模型在科学推理任务上的性能。
摘要：
科学推理对于培养人工智能科学家以及支持人类研究者推进自然科学发现的前沿至关重要。然而，开源社区主要聚焦于数学和编程，而忽视了科学领域，这在很大程度上是由于缺乏开放、大规模、高质量、可验证的科学推理数据集。为了弥合这一差距，我们首先提出了TextbookReasoning，这是一个开放数据集，包含从12,000本大学水平科学教科书中提取的真实参考答案，涵盖7个科学学科的650,000道推理题。我们进一步引入了MegaScience，这是一个大规模的高质量开源数据集混合体，总计1,250,000个实例，通过系统的消融研究开发而成，这些研究评估了各种数据选择方法，以确定每个公开可用的科学数据集的最优子集。同时，我们构建了一个全面的评估系统，覆盖15个基准中的不同学科和问题类型，并整合了全面的答案提取策略以确保评估指标的准确性。我们的实验表明，与现有的开源科学数据集相比，我们的数据集在更简洁的回答长度下实现了更优越的性能和训练效率。此外，我们在MegaScience上训练了Llama3.1、Qwen2.5和Qwen3系列基础模型，这些模型在平均性能上显著优于相应的官方指令模型。此外，MegaScience对更大更强的模型表现出更高的有效性，表明科学调优具有规模效益。我们向社区发布了我们的数据整理流程、评估系统、数据集和七个训练好的模型，以推动科学推理研究。
英文摘要：
Scientific reasoning is critical for developing AI scientists and supporting human researchers in advancing the frontiers of natural science discovery. However, the open-source community has primarily focused on mathematics and coding while neglecting the scientific domain, largely due to the absence of open, large-scale, high-quality, verifiable scientific reasoning datasets. To bridge this gap, we first present TextbookReasoning, an open dataset featuring truthful reference answers extracted from 12k university-level scientific textbooks, comprising 650k reasoning questions spanning 7 scientific disciplines. We further introduce MegaScience, a large-scale mixture of high-quality open-source datasets totaling 1.25 million instances, developed through systematic ablation studies that evaluate various data selection methodologies to identify the optimal subset for each publicly available scientific dataset. Meanwhile, we build a comprehensive evaluation system covering diverse subjects and question types across 15 benchmarks, incorporating comprehensive answer extraction strategies to ensure accurate evaluation metrics. Our experiments demonstrate that our datasets achieve superior performance and training efficiency with more concise response lengths compared to existing open-source scientific datasets. Furthermore, we train Llama3.1, Qwen2.5, and Qwen3 series base models on MegaScience, which significantly outperform the corresponding official instruct models in average performance. In addition, MegaScience exhibits greater effectiveness for larger and stronger models, suggesting a scaling benefit for scientific tuning. We release our data curation pipeline, evaluation system, datasets, and seven trained models to the community to advance scientific reasoning research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
