Large Language Models (LLMs) are reported to hold undesirable attestation bias on inference tasks: when asked to predict if a premise P entails a hypothesis H, instead of considering H's conditional truthfulness entailed by P, LLMs tend to use the out-of-context truth label of H as a fragile proxy. In this paper, we propose a pipeline that exploits this bias to do explicit inductive inference. Our pipeline uses an LLM to transform a premise into a set of attested alternatives, and then aggregate answers of the derived new entailment inquiries to support the original inference prediction. On a directional predicate entailment benchmark, we demonstrate that by applying this simple pipeline, we can improve the overall performance of LLMs on inference and substantially alleviate the impact of their attestation bias.
Shaping Shared Languages: Human and Large Language Models' Inductive Biases in Emergent Communication
Leiden Institute of Advanced Computer Science, Leiden University, Netherlands(莱顿高级计算机科学研究所,莱顿大学,荷兰);School of Computing, University of Kent, United Kingdom(计算学院,肯特大学,英国);Cognitive Psychology Unit, Institute of Psychology, Leiden, the Netherlands(认知心理学单元,心理学研究所,莱顿,荷兰)
A Survey of Inductive Reasoning for Large Language Models
East China Normal University(华东师范大学);Shanghai Innovation Institute(上海创新研究院);Fudan University(复旦大学);Xi’an Jiaotong University(西安交通大学);Shanghai AI Laboratory(上海人工智能实验室)
MIRAGE: Evaluating and Explaining Inductive Reasoning Process in Language Models
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)
Computer Science University of Waterloo(滑铁卢大学计算机科学系);Department of Computer Science Cornell University(康奈尔大学计算机科学系);Department of Computer Science Dalhousie University(达尔豪斯大学计算机科学系);Department of Psychology University of Toronto(多伦多大学心理学系)
On the Inductive Bias of Masked Language Modeling: From Statistical to Syntactic Dependencies
Stanford University(斯坦福大学)
我们研究了如何以无监督的方式掩蔽和预测令牌可以产生语言结构和下游性能增益。最近的理论表明,预训练的语言模型通过隐含地充当下游任务的完形填空缩减的掩码来获得有用的归纳偏差。虽然有吸引力,我们表明,在实践中使用的随机掩蔽策略的成功,不能解释这样的完形填空式的面具。我们使用三个不同分类数据集的特定任务词典构建了类似完形填空的掩码,并表明大多数预训练的性能增益来自与词典无关的通用掩码。为了解释这些通用面具的经验成功,我们证明了掩蔽语言模型(MLM)的目标和现有的方法之间的对应关系,学习统计依赖的图形模型。使用此,我们推导出一种方法,用于提取这些学习的统计依赖MLMs,并表明这些依赖编码有用的归纳偏见的形式的句法结构。在无监督解析评估中,简单地在隐含的统计依赖结构上形成最小生成树优于无监督解析的经典方法(58.74 vs. 55.91 UUAS)。
Codebook Reduction and Saturation: Novel observations on Inductive Thematic Saturation for Large Language Models and initial coding in Thematic Analysis
Abertay University(阿伯泰大学);Yale Medical School(耶鲁医学院)
Knowledge Reasoning Language Model: Unifying Knowledge and Language for Inductive Knowledge Graph Reasoning
The Key Laboratory of Knowledge Engineering with Big Data (the Ministry of Education of China), Hefei University of Technology, China(合肥工业大学大数据知识工程教育部重点实验室);School of Computer Science and Information Engineering, Hefei University of Technology, China(合肥工业大学计算机与信息学院);Nanjing University of Science and Technology, China(南京理工大学);China Unicom Digital Technology Co., Ltd., Beijing, China(联通数字科技有限公司);China Unicom Internet of Things Co., Ltd., Nanjing, China(联通物联网有限责任公司);Shandong Inspur Science Research Institute, Jinan, China(山东浪潮科学研究院);Griffith University, Australia(格里菲斯大学)
Reasoning Can Hurt the Inductive Abilities of Large Language Models
School of Information Sciences University of Illinois at Urbana-Champaign(信息科学学院伊利诺伊大学厄巴纳-香槟分校);Computer Science and Engineering HKUST(计算机科学与工程香港科技大学);Research Scientist, Intel Labs(英特尔实验室研究员);School of Information Sciences University of Illinois Urbana-Champaign(信息科学学院伊利诺伊大学厄巴纳-香槟分校)
Code-driven Number Sequence Calculation: Enhancing the inductive Reasoning Abilities of Large Language Models
East China Normal University(东华大学);Fudan University(复旦大学);Shanghai Innovation Institute(上海创新研究院);Xi’an Jiaotong University(西安交通大学);Shanghai AI Laboratory(上海人工智能实验室)
The Role of Deductive and Inductive Reasoning in Large Language Models
University of Edinburgh(爱丁堡大学);University of Manchester(曼彻斯特大学);University of Washington(华盛顿大学);Tsinghua University(清华大学);Skai Intelligence(Skai智能科技);University of Copenhagen(哥本哈根大学)
The Sword, Shield, and Achilles' Heel: Characterizing the Linguistic Inductive Bias of Large Language Models for Spatial Reasoning in Navigation Planning
East China Normal University(东华大学);Information Engineering University(信息工程大学);Zhengzhou University(郑州大学)
Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinement
Massachusetts Institute of Technology(麻省理工学院);Allen Institute for Artificial Intelligence(艾伦人工智能研究所);University of Washington(华盛顿大学);University of Southern California(南加州大学)