MASSW：面向AI辅助科研工作流的新数据集与基准任务
MASSW: A New Dataset and Benchmark Tasks for AI-Assisted Scientific Workflows
AI总结：
本文提出MASSW数据集，从152,000余篇计算机科学论文中利用大语言模型自动提取科研工作流五方面结构化摘要，并构建多个基准任务以支持AI辅助科研创新。
摘要：
科学创新依赖于详细的工作流，其中包括分析文献、产生想法、验证这些想法、解释结果以及启发后续研究等关键步骤。然而，记录这些工作流的科学出版物篇幅庞大且缺乏结构化，这使得人类研究者和AI系统都难以有效导航和探索科学创新的空间。为了解决这一问题，我们引入了MASSW，一个关于科研工作流多方面摘要的综合文本数据集。MASSW包含超过152,000篇来自过去50年间17个顶级计算机科学会议的同侪评审出版物。利用大语言模型（LLMs），我们自动从这些出版物中提取五个核心方面——背景、关键思想、方法、结果和预期影响——它们对应研究工作流中的五个关键步骤。这些结构化摘要促进了多种下游任务和分析。通过将LLM提取的摘要与人工标注进行比较，验证了其质量。我们通过多个新颖的机器学习任务展示了MASSW的实用性，这些任务可以利用这一新数据集进行基准测试，并沿科研工作流进行各种类型的预测和推荐。MASSW具有巨大潜力，可供研究者创建和基准测试新的AI方法，以优化科研工作流并促进该领域的科学创新。我们的数据集在\url{https://github.com/xingjian-zhang/massw}上公开可用。
英文摘要：
Scientific innovation relies on detailed workflows, which include critical steps such as analyzing literature, generating ideas, validating these ideas, interpreting results, and inspiring follow-up research. However, scientific publications that document these workflows are extensive and unstructured. This makes it difficult for both human researchers and AI systems to effectively navigate and explore the space of scientific innovation. To address this issue, we introduce MASSW, a comprehensive text dataset on Multi-Aspect Summarization of Scientific Workflows. MASSW includes more than 152,000 peer-reviewed publications from 17 leading computer science conferences spanning the past 50 years. Using Large Language Models (LLMs), we automatically extract five core aspects from these publications -- context, key idea, method, outcome, and projected impact -- which correspond to five key steps in the research workflow. These structured summaries facilitate a variety of downstream tasks and analyses. The quality of the LLM-extracted summaries is validated by comparing them with human annotations. We demonstrate the utility of MASSW through multiple novel machine-learning tasks that can be benchmarked using this new dataset, which make various types of predictions and recommendations along the scientific workflow. MASSW holds significant potential for researchers to create and benchmark new AI methods for optimizing scientific workflows and fostering scientific innovation in the field. Our dataset is openly available at \url{https://github.com/xingjian-zhang/massw}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
