发表机构
TCS Research（塔塔咨询服务研究院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究利用引用归一化影响力作为反馈信号，通过奖励模型和强化学习训练构思生成器，使其在科学发现中产生更高预期影响力的想法。
AI 中文摘要
科学构思日益由大型语言模型介导，但当前的构思系统通常在可即时评判的代理指标（如新颖性、清晰度和可行性）上进行训练和评估。这留下了一个问题：科学成果被采纳的延迟信号能否用作反馈，以引导模型朝着预期影响力更高的研究方向前进。我们使用引用归一化影响力作为学术采纳的噪声较大但可扩展的代理指标来研究这一问题。我们通过从超过10万篇计算机科学论文中提取目标条件化的构思描述，并为每篇论文分配一个序数、年份归一化的引用标签，构建了一个大规模数据集。随后，我们训练了一个目标条件化的奖励模型，以从研究目标和构思对中预测引用影响力标签，并利用该奖励通过监督微调及随后的强化学习来对齐构思生成器。为减少循环性，我们采用一个留出法、参考锚定的评估协议来评估生成的构思，该协议将模型输出与同一研究目标下的历史构思进行比较，并根据参考构思的引用影响力标签对判断进行加权。实验表明，我们经过强化学习调优的模型始终能产生比基础模型和监督微调基线具有更高估计影响力的构思。我们的发现将科学影响力定位为一种实用的、以结果为基础的反馈信号，用于在开放式科学发现中对齐大型语言模型。
英文摘要
Scientific ideation is increasingly mediated by large language models, but current ideation systems are usually trained and evaluated on immediately judgeable proxies such as novelty, clarity, and feasibility. This leaves open whether delayed signals of scientific uptake can be used as feedback for steering models toward research directions with higher expected \emph{impact}. We study this question using citation-normalized impact as a noisy but scalable proxy for scholarly uptake. We construct a large-scale dataset from over 100K computer science papers by extracting goal-conditioned idea descriptions and assigning each paper an ordinal, year-normalized citation label. We then train a goal-conditioned reward model to predict citation-impact labels from research goal and idea pairs, and use this reward to align an idea generator through supervised fine-tuning followed by reinforcement learning. To reduce circularity, we evaluate generated ideas with a held-out, reference-grounded protocol that compares model outputs against historical ideas under the same research goal and weights judgments by the reference idea's citation-impact label. Experiments show that our RL-tuned model consistently produces ideas with higher estimated impact than both the base model and supervised fine-tuning baselines. Our findings position scientific impact as a practical, outcome-grounded feedback signal for aligning LLMs in open-ended scientific discovery.
CommentsRLxF Workshop ICML 2026
