arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.21728cs.AIcs.CL
收藏

EvoIdeator:通过基于检查表的强化学习进化科学思想

EvoIdeator: Evolving Scientific Ideas through Checklist-Grounded Reinforcement Learning

Huawei Technologies Co., Ltd.(华为技术有限公司);Vrije Universiteit Amsterdam(荷兰瓦赫宁根大学)

Andreas Sauter, Yuyue Zhao, Jacopo Urbani, Wenxiang Hu, Zaiqiao Meng, Lun Zhou, Xiaohui Yan, Yougang Lyu

AI总结:

本文提出EvoIdeator框架,通过基于检查表的反馈与强化学习结合,提升大语言模型生成高质量科研提案的能力,实验表明其在关键科学指标上优于其他模型。

摘要:

科学思想生成是自主知识发现的核心,但将初始概念转化为高质量研究提案的迭代进化对大语言模型(LLMs)而言是一个严峻挑战。现有强化学习(RL)范式通常依赖基于评分标准的标量奖励,提供全局质量评分但缺乏可操作的细粒度。相反,基于语言的细化方法通常局限于推理时间提示,针对未被显式优化以内化此类批评的模型。为弥合这一差距,我们提出EvoIdeator框架,通过将RL训练目标与基于检查表的反馈对齐,促进科学思想的进化。EvoIdeator利用结构化判断模型生成两种协同信号:(1)用于多维优化的词典奖励,以及(2)提供关于基础性、可行性和方法论严谨性的跨度级语言反馈。通过将这些信号整合到RL循环中,我们使策略能够系统利用精确反馈在优化和推理过程中。广泛实验表明,基于Qwen3-4B构建的EvoIdeator在关键科学指标上显著优于其他大型前沿模型。关键的是,所学的策略能够很好地泛化到多种外部反馈源,无需进一步微调,提供了一条可扩展且严谨的自我完善自主构思路径。

英文摘要:

Scientific idea generation is a cornerstone of autonomous knowledge discovery, yet the iterative evolution required to transform initial concepts into high-quality research proposals remains a formidable challenge for Large Language Models (LLMs). Existing Reinforcement Learning (RL) paradigms often rely on rubric-based scalar rewards that provide global quality scores but lack actionable granularity. Conversely, language-based refinement methods are typically confined to inference-time prompting, targeting models that are not explicitly optimized to internalize such critiques. To bridge this gap, we propose \textbf{EvoIdeator}, a framework that facilitates the evolution of scientific ideas by aligning the RL training objective with \textbf{checklist-grounded feedback}. EvoIdeator leverages a structured judge model to generate two synergistic signals: (1) \emph{lexicographic rewards} for multi-dimensional optimization, and (2) \emph{fine-grained language feedback} that offers span-level critiques regarding grounding, feasibility, and methodological rigor. By integrating these signals into the RL loop, we condition the policy to systematically utilize precise feedback during both optimization and inference. Extensive experiments demonstrate that EvoIdeator, built on Qwen3-4B, significantly outperforms much larger frontier models across key scientific metrics. Crucially, the learned policy exhibits strong generalization to diverse external feedback sources without further fine-tuning, offering a scalable and rigorous path toward self-refining autonomous ideation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑