Despite significant advancements in large language models (LLMs), the rapid and frequent integration of small-scale experiences, such as interactions with surrounding objects, remains a substantial challenge. Two critical factors in assimilating these experiences are (1) Efficacy: the ability to accurately remember recent events; (2) Retention: the capacity to recall long-past experiences. Current methods either embed experiences within model parameters using continual learning, model editing, or knowledge distillation techniques, which often struggle with rapid updates and complex interactions, or rely on external storage to achieve long-term retention, thereby increasing storage requirements. In this paper, we propose SELF-PARAM (Self-Updatable Large Language Models with Parameter Integration). SELF-PARAM requires no extra parameters while ensuring near-optimal efficacy and long-term retention. Our method employs a training objective that minimizes the Kullback-Leibler (KL) divergence between the predictions of an original model (with access to contextual information) and a target model (without such access). By generating diverse question-answer pairs related to the knowledge and minimizing the KL divergence across this dataset, we update the target model to internalize the knowledge seamlessly within its parameters. Evaluations on question-answering and conversational recommendation tasks demonstrate that SELF-PARAM significantly outperforms existing methods, even when accounting for non-zero storage requirements. This advancement paves the way for more efficient and scalable integration of experiences in large language models by embedding knowledge directly into model parameters.
Text-to-SQL as Dual-State Reasoning: Integrating Adaptive Context and Progressive Generation
School of Computer Science, Guangdong University of Technology(广东工业大学计算机科学学院);College of Science, Shantou University(汕头大学科学学院);Peng Cheng Laboratory(鹏城实验室)
Contextual information integration for stance detection via cross-attention
Technical University of Darmstadt(达姆施塔特工业大学);Hessian Center for AI (hessian.AI)(黑森人工智能中心);Lucerne University of Applied Sciences and Arts(卢塞恩应用科学与艺术大学)
LLM-as-RNN: A Recurrent Language Model for Memory Updates and Sequence Prediction
Georgia Institute of Technology(佐治亚理工学院);Peking University(北京大学);Shandong University(山东大学);Huazhong University of Science and Technology(华中科技大学);UT Southwestern Medical Center(西南医学中心)
NOWJ@COLIEE 2025: A Multi-stage Framework Integrating Embedding Models and Large Language Models for Legal Retrieval and Entailment
VNU University of Engineering and Technology(越南工程大学);Japan Advanced Institute of Science and Technology(日本先进科学研究院);Center for Juris-Informatics, ROIS-DS(法律信息中心,ROIS-DS);Research and Development Center for Large Language Models, NII(大型语言模型研发中心,NII)
Can LLMs Reason Over Non-Text Modalities in a Training-Free Manner? A Case Study with In-Context Representation Learning
College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算机与数据科学学院);AI-X, Interdisciplinary Graduate Programme, Nanyang Technological University(南洋理工大学人工智能交叉研究生项目);Lee Kong Chian School of Medicine, Nanyang Technological University(南洋理工大学李科钦医学院);Centre of AI in Medicine (C-AIM), Nanyang Technological University(南洋理工大学医学人工智能中心);University of Toronto(多伦多大学);Brigham and Women’s Hospital, Harvard Medical School(哈佛医学院布里洛妇女医院);Massachusetts Institute of Technology(麻省理工学院)
Ontology Memory-Augmented ASR Correction for Long Text-Speech Interleaved Conversations
Institute of Computing and Intelligence, Harbin Institute of Technology (Shenzhen), China(哈尔滨工业大学(深圳)计算与智能研究所);Shenzhen Loop Area Institute (SLAI), China(深圳环域研究所)
National University of Singapore(新加坡国立大学);University of Sussex(苏塞克斯大学);Wuhan University(武汉大学);University of Texas at Dallas(德克萨斯大学达拉斯分校);Nanyang Technological University(南洋理工大学);Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳));University of Illinois at Chicago(伊利诺伊大学香槟分校)
Agentic AI for Substance Use Education: Integrating Regulatory and Scientific Knowledge Sources
Department of Social Science, Texas Woman's University(德克萨斯女子大学社会科学系);Department of Computer Science, Augsburg University of Applied Sciences(应用科学大学计算机科学系);School of Computer Science, University of Oklahoma(俄克拉荷马大学计算机科学学院)
CausalRAG: Integrating Causal Graphs into Retrieval-Augmented Generation
Department of Computer and Data Sciences, Case Western Reserve University(计算机与数据科学系,凯斯西储大学);Department of Design and Innovation, Case Western Reserve University(设计与创新系,凯斯西储大学)
Graph Your Way to Inspiration: Integrating Co-Author Graphs with Retrieval-Augmented Generation for Large Language Model Based Scientific Idea Generation
School of Computing Newcastle University(计算学院新castle大学)
Contextualized Structural Self-supervised Learning for Ontology Matching
Peoples' Friendship University of Russia (RUDN University)(俄罗斯人民友谊大学(卢蒙巴大学));Joint Institute for Nuclear Research(联合核研究所);Vrije Universiteit Amsterdam(阿姆斯特丹自由大学);University of Skövde(舍夫德大学);University of Illinois at Chicago(伊利诺伊大学芝加哥分校)