Adapter-based Approaches to Knowledge-enhanced Language Models -- A Survey
Technical University of Munich(慕尼黑工业大学);SEBIS, Technical University of Munich(慕尼黑工业大学SEBIS)
Alexander Fichtl, Juraj Vladika, Georg Groh
备注:
12 pages, 4 figures. Published at KEOD24 via SciTePress
发表信息:
In Proceedings of the 16th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - KEOD 2024; ISBN 978-989-758-716-0; ISSN 2184-3228, SciTePress, pages 95-107
Knowledge-enhanced language models (KELMs) have emerged as promising tools to bridge the gap between large-scale language models and domain-specific knowledge. KELMs can achieve higher factual accuracy and mitigate hallucinations by leveraging knowledge graphs (KGs). They are frequently combined with adapter modules to reduce the computational load and risk of catastrophic forgetting. In this paper, we conduct a systematic literature review (SLR) on adapter-based approaches to KELMs. We provide a structured overview of existing methodologies in the field through quantitative and qualitative analysis and explore the strengths and potential shortcomings of individual approaches. We show that general knowledge and domain-specific approaches have been frequently explored along with various adapter architectures and downstream tasks. We particularly focused on the popular biomedical domain, where we provided an insightful performance comparison of existing KELMs. We outline the main trends and propose promising future directions.
Context-Aware Adapter Tuning for Few-Shot Relation Learning in Knowledge Graphs
School of Computing and Information Systems, Singapore Management University(新加坡管理大学计算与信息系统学院);Institute for Infocomm Research, A*STAR(新加坡科技研究局信息通信研究所);A*STAR Centre for Frontier AI Research(新加坡科技研究局前沿人工智能研究中心)
Large Language Model Enhanced Knowledge Representation Learning: A Survey
College of Intelligence and Computing, Tianjin University(天津大学智能与计算学院);College of Design and Innovation, Tongji University(同济大学设计与创新学院);Faculty of Science and Technology, University of Macao(澳门大学科技学院)
Lexicon Enhanced Chinese Sequence Labeling Using BERT Adapter
DAMO Academy, Alibaba Group(阿里巴巴达摩院);College of Computer Science, Nankai University(南开大学计算机学院);School of Engineering, Westlake University(西湖大学工学院);Institute of Advanced Technology, Westlake Institute for Advanced Study(西湖高等研究院前沿技术研究所)
MedualTime: A Dual-Adapter Language Model for Medical Time Series-Text Multimodal Learning
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学);Technical University of Munich(慕尼黑技术大学);Columbia University(哥伦比亚大学)
Evolution of meta's llama models and parameter-efficient fine-tuning of large language models: a survey
University of Kurdistan(库尔德斯坦大学);Queen Mary University(女王玛丽大学);Torrens University Australia(澳大利亚托伦斯大学);University of Technology Sydney(悉尼技术大学);University of Kurdistan Sanandaj, Iran(伊朗桑和杰德大学);TU Dortmund University(多特蒙德技术大学);University of Kurdistan Hewler(库尔德斯坦大学海勒);Tehran University(德黑兰大学)
University of Notre Dame(圣母大学);Microsoft Research(微软研究院);The Chinese University of Hong Kong(香港中文大学);University of California at San Diego(加州大学圣迭戈分校);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
Polypersona: Persona-Grounded LLM for Synthetic Survey Responses
George Mason University, Virginia, USA(乔治·马歇尔大学);New Jersey Institute of Technology (NJIT), New Jersey, USA(新泽西理工学院);University of Southern Queensland, Queensland, Australia(南方昆士兰大学);Edinburgh Napier University, Edinburgh, Scotland(爱丁堡纳皮尔大学)
SGSimEval: A Comprehensive Multifaceted and Similarity-Enhanced Benchmark for Automatic Survey Generation Systems
The Hong Kong Polytechnic University(香港理工大学);The Education University of Hong Kong(香港教育大学);China Mobile (Hong Kong) Innovation and Research Institute(中国移动(香港)创新与研究院);Lingnan University(岭南大学)
X-LoRA: Mixture of Low-Rank Adapter Experts, a Flexible Framework for Large Language Models with Applications in Protein Mechanics and Molecular Design
Academy for Science and Design (ASD)(科学设计学院);Massachusetts Institute of Technology (MIT)(麻省理工学院)
Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methods
The Chinese University of Hong Kong(香港中文大学);The Hong Kong Polytechnic University(香港理工大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Shenzhen Institute of Artificial Intelligence and Robotics for Society (AIRS)(深圳市人工智能与机器人研究院);Guangdong-Hongkong-Macao Greater Bay Area Weather Research Center for Monitoring Warning and Forecasting (Shenzhen Institute of Meteorological Innovation)(粤港澳大湾区天气监测预警预报研究中心(深圳市气象创新研究院));Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China(电子科技大学深圳高等研究院)
A Survey of Knowledge Enhanced Pre-trained Language Models
Beijing Institute of Technology(北京理工大学);Beijing University of Posts and Telecommunications(北京邮电大学);Tsinghua University(清华大学);Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳))
Injecting Domain-Specific Knowledge into Large Language Models: A Comprehensive Survey
Mohamed bin Zayed University of Artificial Intelligence(Mohamed bin Zayed大学);Northeastern University(东北大学);Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学人工智能学院);ByteDance(字节跳动);Wuhan University(武汉大学)