arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2311.05965cs.CL
收藏

大型语言模型是零样本假设提出者

Large Language Models are Zero Shot Hypothesis Proposers

Tsinghua University(清华大学);Harbin Institute of Technology(哈尔滨工业大学);University of Washington(华盛顿大学);The First Affiliated Hospital of Nanchang University(南昌大学第一附属医院)

Biqing Qi, Kaiyan Zhang, Haoxiang Li, Kai Tian, Sihang Zeng, Zhang-Ren Chen, Bowen Zhou

AI总结:

本文研究大型语言模型能否零样本提出科学假设,构建生物医学数据集并评估多种模型,提出多智能体框架和四个评估指标,发现LLMs能生成已验证假设且不确定性有助于生成,支持其作为科学发现催化剂。

摘要:

重大科学发现推动了人类文明的进步。科学文献和数据的爆炸式增长造成了跨学科的信息壁垒,这些壁垒减缓了科学发现的步伐。大型语言模型(LLMs)拥有丰富的全球性和跨学科知识,有望打破这些信息壁垒,促进新一轮科学发现。然而,LLMs在科学发现方面的潜力尚未被正式探索。在本文中,我们从调查LLMs是否能提出科学假设开始。为此,我们构建了一个由生物医学文献中的背景知识和假设对组成的数据集。该数据集根据出版日期分为训练集、已见测试集和未见测试集,以控制可见性。随后,我们在零样本、少样本和微调设置下评估了各种顶级指令模型(包括闭源和开源LLMs)的假设生成能力。此外,我们引入了一个基于LLM的多智能体协作框架,该框架具有不同的角色设计和外部工具,以增强与生成假设相关的能力。我们还通过全面审查设计了四个指标,以评估基于ChatGPT和人工评估的生成假设。通过实验和分析,我们得出以下发现:1)LLMs令人惊讶地从测试文献中生成未经训练但已验证的假设。2)增加不确定性有助于候选生成,可能增强零样本假设生成能力。这些发现有力地支持了LLMs作为新科学发现催化剂的潜力,并为进一步探索提供了指导。

英文摘要:

Significant scientific discoveries have driven the progress of human civilisation. The explosion of scientific literature and data has created information barriers across disciplines that have slowed the pace of scientific discovery. Large Language Models (LLMs) hold a wealth of global and interdisciplinary knowledge that promises to break down these information barriers and foster a new wave of scientific discovery. However, the potential of LLMs for scientific discovery has not been formally explored. In this paper, we start from investigating whether LLMs can propose scientific hypotheses. To this end, we construct a dataset consist of background knowledge and hypothesis pairs from biomedical literature. The dataset is divided into training, seen, and unseen test sets based on the publication date to control visibility. We subsequently evaluate the hypothesis generation capabilities of various top-tier instructed models in zero-shot, few-shot, and fine-tuning settings, including both closed and open-source LLMs. Additionally, we introduce an LLM-based multi-agent cooperative framework with different role designs and external tools to enhance the capabilities related to generating hypotheses. We also design four metrics through a comprehensive review to evaluate the generated hypotheses for both ChatGPT-based and human evaluations. Through experiments and analyses, we arrive at the following findings: 1) LLMs surprisingly generate untrained yet validated hypotheses from testing literature. 2) Increasing uncertainty facilitates candidate generation, potentially enhancing zero-shot hypothesis generation capabilities. These findings strongly support the potential of LLMs as catalysts for new scientific discoveries and guide further exploration.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑