ResearchTown:人类研究社区的模拟器
ResearchTown: Simulator of Human Research Community
AI总结:
本文提出ResearchTown多智能体框架和TextGNN文本推理框架,将研究社区建模为智能体-数据图以模拟论文写作、审稿等活动,并构建ResearchBench基准验证其能真实模拟协作研究、稳健处理多研究者多论文场景并生成跨学科研究想法。
摘要:
大型语言模型(LLMs)在科学领域已展现出显著潜力,但一个根本性问题仍未得到解答:我们能否用LLMs模拟人类研究社区?回答这一问题可以加深我们对想法头脑风暴背后过程的理解,并启发新颖科学洞见的自动发现。在本工作中,我们提出ResearchTown,一个用于研究社区模拟的多智能体框架。在该框架内,人类研究社区被简化为一个智能体-数据图,其中研究人员和论文分别表示为智能体类型节点和数据类型节点,并基于它们的合作关系进行连接。我们还引入TextGNN,一个基于文本的推理框架,它将各种研究活动(例如论文阅读、论文写作和审稿意见撰写)建模为智能体-数据图上统一消息传递过程的特殊形式。为了评估研究社区模拟的质量,我们提出ResearchBench,一个使用节点掩码预测任务进行基于相似度的可扩展且客观评估的基准。我们的实验揭示了三个关键发现:(1)ResearchTown能够提供对协作研究活动(包括论文写作和审稿意见撰写)的真实模拟;(2)ResearchTown能够在多个研究人员和多样化论文的情况下保持稳健的模拟;(3)ResearchTown能够生成跨学科研究想法,这些想法有可能启发开创性的研究方向。
英文摘要:
Large Language Models (LLMs) have demonstrated remarkable potential in scientific domains, yet a fundamental question remains unanswered: Can we simulate human research communities with LLMs? Addressing this question can deepen our understanding of the processes behind idea brainstorming and inspire the automatic discovery of novel scientific insights. In this work, we propose ResearchTown, a multi-agent framework for research community simulation. Within this framework, the human research community is simplified as an agent-data graph, where researchers and papers are represented as agent-type and data-type nodes, respectively, and connected based on their collaboration relationships. We also introduce TextGNN, a text-based inference framework that models various research activities (e.g., paper reading, paper writing, and review writing) as special forms of a unified message-passing process on the agent-data graph. To evaluate the quality of the research community simulation, we present ResearchBench, a benchmark that uses a node-masking prediction task for scalable and objective assessment based on similarity. Our experiments reveal three key findings: (1) ResearchTown can provide a realistic simulation of collaborative research activities, including paper writing and review writing; (2) ResearchTown can maintain robust simulation with multiple researchers and diverse papers; (3) ResearchTown can generate interdisciplinary research ideas that potentially inspire pioneering research directions.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。