Large Language Models are Zero Shot Hypothesis Proposers
Tsinghua University(清华大学);Harbin Institute of Technology(哈尔滨工业大学);University of Washington(华盛顿大学);The First Affiliated Hospital of Nanchang University(南昌大学第一附属医院)
Significant scientific discoveries have driven the progress of human civilisation. The explosion of scientific literature and data has created information barriers across disciplines that have slowed the pace of scientific discovery. Large Language Models (LLMs) hold a wealth of global and interdisciplinary knowledge that promises to break down these information barriers and foster a new wave of scientific discovery. However, the potential of LLMs for scientific discovery has not been formally explored. In this paper, we start from investigating whether LLMs can propose scientific hypotheses. To this end, we construct a dataset consist of background knowledge and hypothesis pairs from biomedical literature. The dataset is divided into training, seen, and unseen test sets based on the publication date to control visibility. We subsequently evaluate the hypothesis generation capabilities of various top-tier instructed models in zero-shot, few-shot, and fine-tuning settings, including both closed and open-source LLMs. Additionally, we introduce an LLM-based multi-agent cooperative framework with different role designs and external tools to enhance the capabilities related to generating hypotheses. We also design four metrics through a comprehensive review to evaluate the generated hypotheses for both ChatGPT-based and human evaluations. Through experiments and analyses, we arrive at the following findings: 1) LLMs surprisingly generate untrained yet validated hypotheses from testing literature. 2) Increasing uncertainty facilitates candidate generation, potentially enhancing zero-shot hypothesis generation capabilities. These findings strongly support the potential of LLMs as catalysts for new scientific discoveries and guide further exploration.
Intercepting the Kangaroo: Experimental Astrolinguistics with Constructed Lexicons, Active Probing, and Large Language Models as Informants and Hypothesis Proposers
Large Language Models as Biomedical Hypothesis Generators: A Comprehensive Evaluation
Tsinghua University(清华大学);Shanghai AI Laboratory(上海人工智能实验室);Harbin Institute of Technology(哈尔滨工业大学);University of Washington(华盛顿大学);The First Affiliated Hospital of Nanchang University(南昌大学第一附属医院)
Analysis of Error Sources in LLM-based Hypothesis Search for Few-Shot Rule Induction
Department of Statistics, University of California, Los Angeles(统计学系,加州大学洛杉矶分校);Microsoft, Redmond, WA(微软公司,西雅图,华盛顿州);Department of Psychology, University of California, Los Angeles(心理学系,加州大学洛杉矶分校)
HyKGE: A Hypothesis Knowledge Graph Enhanced Framework for Accurate and Reliable Medical LLMs Responses
Key Laboratory of High Confidence Software Technologies (Peking University)(高可信软件技术重点实验室(北京大学));School of Computer Science, Peking University(北京大学计算机学院)
REFLEX: Metacognitive Reasoning for Reflective Zero-Shot Robotic Planning with Large Language Models
School of Applied and Creative Computing, Purdue University(应用与创意计算学院,普渡大学);Bowen School of Construction, Purdue University(建设学院,普渡大学);School of Engineering Technology, Purdue University(工程技术学院,普渡大学);Department of Technology Leadership and Innovation, Purdue University(技术领导与创新部门,普渡大学)
TEGEE: Task dEfinition Guided Expert Ensembling for Generalizable and Few-shot Learning
HKUST(香港科技大学);University of Manchester(曼彻斯特大学);Institute of Automation, CAS(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);M-A-P;Beijing University of Posts and Telecommunications(北京邮电大学);Ohio State University(俄亥俄州立大学);University of Waterloo(滑铁卢大学);Peking University(北京大学);BAAI(北京智源人工智能研究院)
Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinement
Massachusetts Institute of Technology(麻省理工学院);Allen Institute for Artificial Intelligence(艾伦人工智能研究所);University of Washington(华盛顿大学);University of Southern California(南加州大学)
AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languages
University of Colorado Boulder(科罗拉多大学博尔德分校);University of Stuttgart(斯图加特大学);University of Edinburgh(爱丁堡大学);Microsoft Turing(微软图灵);Facebook AI Research(脸书人工智能研究院);New York University(纽约大学);Universidad de la República, Uruguay(乌拉圭共和国大学);Universidad Tecnológica de Tlaxcala(特拉斯卡拉理工大学);University of Zurich(苏黎世大学);Carnegie Mellon University(卡内基梅隆大学);Universidad Nacional Autónoma de México(墨西哥国立自治大学);Universidade Tecnológica Federal do Paraná(巴拉那联邦理工大学);Dartmouth College(达特茅斯学院)
Few-shot Language Coordination by Modeling Theory of Mind
Carnegie Mellon University(卡内基梅隆大学)
$\textit{No man is an island.}$人类通过在简短的对话中与不同的对话者进行协调来与大型社区进行交流。这种能力已经被构建神经通信代理的研究所忽视。我们研究了Few-Shot $\textit{language coordination}$的任务:智能体快速适应他们的会话伙伴的语言能力。与目前的自我游戏训练的沟通代理不同,我们需要领导代理与具有不同语言能力的代理$\textit{population}$协调,快速适应与人口中看不见的代理进行沟通。这就需要有能力模仿伴侣的信念,这是人类沟通的重要组成部分。从心理理论(ToM; Premack& Woodruff(1978))中获得灵感,我们研究了说话者明确地模拟听者的心理状态的效果。正如我们的实验所示,说话者获得了预测其伙伴反应的能力,这有助于它生成简明表达其交际目标的指令。我们研究了我们的假设,即与心理理论建模生成的指令产生更好的通信性能在参考游戏和语言导航任务。从我们的实验中的积极结果暗示的重要性,明确建模通信作为一个社会语用的进步。