Learn More, Forget Less: A Gradient-Aware Data Selection Approach for LLM
Fu Foundation School of Engineering and Applied Science, Columbia University(哥伦比亚大学工程与应用科学学院);School of Computer Science and Engineering, Beihang University(北航计算机科学与工程学院)
Despite large language models (LLMs) have achieved impressive achievements across numerous tasks, supervised fine-tuning (SFT) remains essential for adapting these models to specialized domains. However, SFT for domain specialization can be resource-intensive and sometimes leads to a deterioration in performance over general capabilities due to catastrophic forgetting (CF). To address these issues, we propose a self-adaptive gradient-aware data selection approach (GrADS) for supervised fine-tuning of LLMs, which identifies effective subsets of training data by analyzing gradients obtained from a preliminary training phase. Specifically, we design self-guided criteria that leverage the magnitude and statistical distribution of gradients to prioritize examples that contribute the most to the model's learning process. This approach enables the acquisition of representative samples that enhance LLMs understanding of domain-specific tasks. Through extensive experimentation with various LLMs across diverse domains such as medicine, law, and finance, GrADS has demonstrated significant efficiency and cost-effectiveness. Remarkably, utilizing merely 5% of the selected GrADS data, LLMs already surpass the performance of those fine-tuned on the entire dataset, and increasing to 50% of the data results in significant improvements! With catastrophic forgetting substantially mitigated simultaneously. We will release our code for GrADS later.
AWARe: Mitigating Catastrophic Forgetting via Activation-Weighted Adaptive REtention
School of Software Technology, Zhejiang University(浙江大学软件学院);College of Intelligent Robotics and Advanced Manufacturing, Fudan University(复旦大学智能机器人与先进制造学院);School of Artificial Intelligence, Sun Yat-Sen University(中山大学人工智能学院);Graduate School of Information Science, Hokkaido University(北海道大学信息科学研究院)
Dynamic Context-oriented Decomposition for Task-aware Low-rank Adaptation with Less Forgetting and Faster Convergence
King Abdullah University of Science and Technology, Saudi Arabia(卡布尔大学);University of California, Merced, USA(加州大学梅尔德分校);University of Oxford, U.K.(牛津大学)
CURE:Circuit-Aware Unlearning for LLM-based Recommendation
University of Massachusetts Lowell(马萨诸塞大学洛厄尔分校);Zhejiang University(浙江大学);The Chinese University of Hong Kong(香港中文大学);National University of Singapore(新加坡国立大学)
Enhancing Continual Learning for Software Vulnerability Prediction: Addressing Catastrophic Forgetting via Hybrid-Confidence-Aware Selective Replay for Temporal LLM Fine-Tuning
School of Computer Science, University of Nottingham, Nottingham, United Kingdom(诺丁汉大学计算机科学学院);School of Informatics and Computing, Northern Arizona University, Flagstaff, United States of America(北亚利桑那大学信息学院);School of Computer Science, University of Nottingham, Ningbo, China(诺丁汉大学宁波校区)
SuperLocalMemory V3.3: The Living Brain -- Biologically-Inspired Forgetting, Cognitive Quantization, and Multi-Channel Retrieval for Zero-LLM Agent Memory Systems
RECALL: REpresentation-aligned Catastrophic-forgetting ALLeviation via Hierarchical Model Merging
Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Peng Cheng Laboratory(鹏城实验室);Huazhong University of Science and Technology(华中科技大学);Xiamen University(厦门大学);The Hong Kong University of Science and Technology, Guangzhou(香港科技大学(广州));School of Biomedical Engineering, Tsinghua University(清华大学生物医学工程学院)
TRACE: Discovering Task-Specific Parameter via Adaptation-Aware Probing for Continual Fine-Tuning
College of Computer Science and Technology, Jilin University(吉林大学计算机科学与技术学院);College of Software, Jilin University(吉林大学软件学院);Fudan University(复旦大学);School of Mathematical and Computer Sciences, Heriot-Watt University(赫瑞-瓦特大学数学与计算机科学学院);Department of Information Engineering and Computer Science, University of Trento(特伦托大学信息工程与计算机科学系)
Indian Institute of Information Technology, Bhopal, India(印度比哈尔理工学院);University of California, Santa Cruz, USA(加州大学圣克鲁兹分校);Independent Researcher(独立研究者);Stanford University, USA(斯坦福大学);BITS Pilani Goa, India(比斯拉米印度学院)
Continual Machine Reading Comprehension via Uncertainty-aware Fixed Memory and Adversarial Domain Adaptation
Tsinghua University(清华大学);Beijing National Research Center for Information Science and Technology(北京信息科学与技术国家研究中心);Hebei University of Science and Technology(河北科技大学)
Improving Background Based Conversation with Context-aware Knowledge Pre-selection
University of Amsterdam(阿姆斯特丹大学)
基于背景的对话(Background Based Conversations,BBCs)已经被开发以通过利用背景知识来使对话系统生成更多信息且更自然的响应。现有的BBC方法可以分为两类:基于提取的方法和基于生成的方法。前一个摘录来自背景材料,作为不一定自然的反应。后者产生的反应是自然的,但不一定有效地利用背景知识。在本文中,我们专注于基于生成的方法,并提出了一个模型,即上下文感知的知识预选择(CaKe),它引入了一个预选择过程,使用动态的双向注意,以改善知识的选择,使用的话语历史上下文作为先验信息,以选择最相关的背景材料。实验结果表明,我们的模型优于当前最先进的基线,表明它受益于预选过程,从而提高信息量和流畅性。