arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.14634cs.HCcs.AI
收藏

基于领域重构与新颖性评估的人机协同科学构想系统

Human-LLM Compound System for Scientific Ideation through Facet Recombination and Novelty Evaluation

University of Washington(华盛顿大学);Microsoft(微软);Allen Institute for AI(人工智能研究院)

Marissa Radensky, Simra Shahid, Raymond Fok, Pao Siangliulue, Tom Hope, Daniel S. Weld

AI总结:

Scideator通过领域重构与新颖性评估模块,帮助用户在科学构想中生成更具创造性的想法,实验显示其在想法探索和表达性方面优于传统LLM方法。

摘要:

科学构想过程通常涉及将现有论文的要素重新组合以产生新想法。我们提出了Scideator,首个基于要素的科学构想人机协同系统。从用户提供的论文出发,Scideator提取关键要素--目的、机制和评估--并允许用户交互式重新组合要素以合成想法。Scideator由三个设计选择驱动:(1) 人类在回路的要素重新组合,用户从检索的论文中选择要素,系统通过Faceted Idea Generator模块寻找跨要素的类比生成想法;(2) 距离控制检索通过Analogous Paper Facet Finder模块,提供从相同主题到完全不同领域的论文范围;(3) 基于要素的新颖性验证通过Idea Novelty Checker模块,一个检索后重排序流程,帮助用户评估想法的原创性。在计算机科学研究人员的用户研究中,Scideator比使用相同基础LLM但无要素模块的基线提供了显著更多的创造力支持,特别是在想法探索和表达性方面。消融实验进一步表明,要素对新颖性检查器有益:基于要素的检索后重排序比标准检索和重排序显示更多相关论文,且基于要素的新颖性分类器优于基于无结构想法和论文推理的分类器。

英文摘要:

The scientific ideation process often involves blending facets of existing papers to create new ideas. We contribute Scideator, the first human-LLM system for facet-based scientific ideation. Starting from user-provided papers, Scideator extracts key facets -- purposes, mechanisms, and evaluations -- from these and related papers, allowing users to interactively recombine facets to synthesize ideas. Scideator is driven by three design choices: (1) human-in-the-loop facet recombination, in which users select facets from retrieved papers and the system generates ideas by finding analogies across them via the Faceted Idea Generator module; (2) distance-controlled retrieval via the Analogous Paper Facet Finder module, which surfaces papers ranging from the same topic to entirely different areas to provide a spectrum of directions; and (3) facet-based novelty verification via the Idea Novelty Checker module, a retrieve-then-rerank pipeline that helps users to evaluate idea originality using facets. In a user study with computer science researchers, Scideator provided significantly more creativity support than a baseline using the same backbone LLM without our facet-based modules, particularly in idea exploration and expressiveness. Ablations further show that the facets benefit the novelty checker: facet-based retrieve-then-rerank surfaces more relevant papers than standard retrieval and re-ranking, and a facet-grounded novelty classifier outperforms classifiers that reason over unstructured ideas and papers.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑