Understanding and Patching Compositional Reasoning in LLMs
School of Computer Science and Technology, University of Science and Technology of China(中国科学技术大学计算机科学与技术学院);Department of Computer Science, City University of Hong Kong(香港城市大学计算机科学系);City University of Hong Kong Shenzhen Research Institute(香港城市大学深圳研究院);School of Computer Science and Engineering, Nanyang Technological University(南洋理工大学计算机科学与工程学院)
LLMs have marked a revolutonary shift, yet they falter when faced with compositional reasoning tasks. Our research embarks on a quest to uncover the root causes of compositional reasoning failures of LLMs, uncovering that most of them stem from the improperly generated or leveraged implicit reasoning results. Inspired by our empirical findings, we resort to Logit Lens and an intervention experiment to dissect the inner hidden states of LLMs. This deep dive reveals that implicit reasoning results indeed surface within middle layers and play a causative role in shaping the final explicit reasoning results. Our exploration further locates multi-head self-attention (MHSA) modules within these layers, which emerge as the linchpins in accurate generation and leveraing of implicit reasoning results. Grounded on the above findings, we develop CREME, a lightweight method to patch errors in compositional reasoning via editing the located MHSA modules. Our empirical evidence stands testament to CREME's effectiveness, paving the way for autonomously and continuously enhancing compositional reasoning capabilities in language models.
Exploring the Compositional Deficiency of Large Language Models in Mathematical Reasoning
School of Computer Science, Fudan University(复旦大学计算机科学学院);Shanghai Key Laboratory of Intelligent Information Processing, Fudan University(复旦大学上海智能信息处理重点实验室)
Compositional Generalization from Learned Skills via CoT Training: A Theoretical and Structural Analysis for Reasoning
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学人工智能学院);Beijing Key Laboratory of Research on Large Models and Intelligent Governance(北京大型模型与智能治理研究重点实验室);Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MOE(下一代智能搜索与推荐工程研究中心,教育部);Tianjin University of Science and Technology(天津科技大学);Beijing Institute of Technology(北京理工大学)
Faith and Fate: Limits of Transformers on Compositionality
Allen Institute for Artificial Intelligence(艾伦人工智能研究所);University of Washington(华盛顿大学);University of Southern California(南加州大学);University of Chicago(芝加哥大学)
Compositional Reasoning with Transformers, RNNs, and Chain of Thought
Courant Institute of Mathematical Sciences(Courant数学科学研究所);New York University(纽约大学);Center for Data Science(数据科学中心);Center for Computational Mathematics(计算数学中心);Flatiron Institute(Flatiron研究所)
ReaSCAN: Compositional Reasoning in Language Grounding
Stanford University(斯坦福大学);National University of Singapore(新加坡国立大学);IHPC, A*STAR(新加坡科技研究局信息与通信研究所)
将语言组合映射到指称、关系和动作的能力是语言理解的重要组成部分。最近的gSCAN数据集(Ruis et al. 2020,NeurIPS)是一个鼓舞人心的尝试,旨在评估模型在涉及导航指令的场景中学习这种基础的能力。然而,我们表明,gSCAN的高度约束的设计意味着,它不需要组合的解释,它的指令和场景的许多细节是不需要任务的成功。为了解决这些局限性,我们提出了ReaSCAN,这是一个基准数据集,它建立在gSCAN基础上,但需要对实体和关系进行组合语言解释和推理。我们在ReaSCAN上评估了两个模型:多模态基线和最先进的图卷积神经模型。这些实验表明,ReaSCAN对于两种神经架构都比gSCAN要难得多。这表明ReaSCAN可以作为一个有价值的基准,以促进我们对模型的组合泛化和推理能力的理解。
KQA Pro: A Dataset with Explicit Compositional Programs for Complex Question Answering over Knowledge Base
Tsinghua University(清华大学);National University of Singapore(新加坡国立大学);ETH Zürich(苏黎世联邦理工学院);Noah’s Ark Lab, Huawei Technologies(华为技术有限公司诺亚方舟实验室);Nanyang Technological University(南洋理工大学);Cloud BU, Huawei Technologies(华为技术有限公司云业务部)
Complexity Control Facilitates Reasoning-Based Compositional Generalization in Transformers
School of Artificial Intelligence, Shanghai Jiao Tong University(上海交通大学人工智能学院);Institute for Advanced Algorithms Research, Shanghai Seres Information Technology Co., Ltd(上海赛尔斯信息技术有限公司高级算法研究院)
Compositional Language Understanding with Text-based Relational Reasoning
Mila(米拉研究所);McGill University(麦吉尔大学);Université de Montréal(蒙特利尔大学);Facebook AI Research (FAIR)(脸书人工智能研究院);Samsung Advanced Institute of Technology (SAIT)(三星综合技术院)
Compositional Reasoning Depth Predicts Clinical AI Failure: Empirical Evidence Consistent with Transformer Compositionality Limits in Electronic Health Record Question Answering
University of California San Francisco(加州大学旧金山分校);Waymark
Atomic Skills are the Prerequisite: When Reinforcement Learning Synthesizes Compositional Reasoning, and When It Only Amplifies
University of Waterloo(滑铁卢大学);Duke University(杜克大学);National University of Singapore(新加坡国立大学);Princeton University(普林斯顿大学);Peking University(北京大学);University of California, Santa Barbara(加州大学圣巴巴拉分校)