Recall, Retrieve and Reason: Towards Better In-Context Relation Extraction
School of Computer Science and Engineering, Southeast University(东南大学计算机科学与工程学院);Beijing Institute of Computer Technology and Application(北京计算机技术及应用研究所);Interdisciplinary Applications (Southeast University), Ministry of Education(教育部东南大学交叉应用中心)
Guozheng Li, Peng Wang, Wenjun Ke, Yikai Guo, Ke Ji, Ziyu Shang, Jiajun Liu, Zijie Xu
Relation extraction (RE) aims to identify relations between entities mentioned in texts. Although large language models (LLMs) have demonstrated impressive in-context learning (ICL) abilities in various tasks, they still suffer from poor performances compared to most supervised fine-tuned RE methods. Utilizing ICL for RE with LLMs encounters two challenges: (1) retrieving good demonstrations from training examples, and (2) enabling LLMs exhibit strong ICL abilities in RE. On the one hand, retrieving good demonstrations is a non-trivial process in RE, which easily results in low relevance regarding entities and relations. On the other hand, ICL with an LLM achieves poor performance in RE while RE is different from language modeling in nature or the LLM is not large enough. In this work, we propose a novel recall-retrieve-reason RE framework that synergizes LLMs with retrieval corpora (training examples) to enable relevant retrieving and reliable in-context reasoning. Specifically, we distill the consistently ontological knowledge from training datasets to let LLMs generate relevant entity pairs grounded by retrieval corpora as valid queries. These entity pairs are then used to retrieve relevant training examples from the retrieval corpora as demonstrations for LLMs to conduct better ICL via instruction tuning. Extensive experiments on different LLMs and RE datasets demonstrate that our method generates relevant and valid entity pairs and boosts ICL abilities of LLMs, achieving competitive or new state-of-the-art performance on sentence-level RE compared to previous supervised fine-tuning methods and ICL-based methods.
Unveiling Factual Recall Behaviors of Large Language Models through Knowledge Neurons
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Beijing Wenge Technology Co., Ltd(北京文格科技有限公司)
FAIR at Meta(Meta的FAIR);University of Washington(华盛顿大学);National University of Singapore(新加坡国立大学);Singapore-MIT Alliance for Research(新加坡-麻省理工联合研究联盟);Allen Institute for Artificial Intelligence(人工智能研究所);Stanford University(斯坦福大学);Massachusetts Institute of Technology(麻省理工学院);University of California, Berkeley(加州大学伯克利分校)
DeepRAG: Thinking to Retrieve Step by Step for Large Language Models
Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所信息处理实验室);University of Chinese Academy of Sciences(中国科学院大学);Pattern Recognition Center, WeChat AI, Tencent Inc, China(腾讯人工智能部模式识别中心)
Retrieve, Integrate, and Synthesize: Spatial-Semantic Grounded Latent Visual Reasoning
State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, and Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University(人机混合增强智能国家重点实验室,人工智能与机器人研究院,西安交通大学)
KGFR: A Foundation Retriever for Generalized Knowledge Graph Question Answering
School of Computer Science, Nanjing University of Information Science and Technology(南京信息工程大学计算机科学学院);State Key Laboratory for Novel Software Technology, Nanjing University(南京大学软件新技术国家重点实验室);National Institute of Healthcare Data Science, Nanjing University(南京大学健康数据科学国家研究院);Engineering Research Center of Digital Forensics, Ministry of Education, Nanjing University of Information Science and Technology(南京信息工程大学教育部长江数字取证工程研究中心)
Locate-then-edit for Multi-hop Factual Recall under Knowledge Editing
Peking University(北京大学);Provable Responsible AI and Data Analytics (PRADA) Lab(可验证负责任人工智能与数据分析(PRADA)实验室);King Abdullah University of Science and Technology(阿卜杜拉国王科技大学);South China University of Technology(华南理工大学)
ReFIT: Relevance Feedback from a Reranker during Inference
University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Allen Institute for AI(艾伦人工智能研究所);IBM Research AI(IBM人工智能研究院);Yale University(耶鲁大学);University of Washington(华盛顿大学)