Parenting：通过参数解耦与定制微调优化检索增强语言模型的知识选择
Parenting: Optimizing Knowledge Selection of Retrieval-Augmented Language Models with Parameter Decoupling and Tailored Tuning
AI总结：
针对检索增强语言模型内外知识整合缺乏有效控制的问题，提出Parenting框架，通过结合前向和反向传播信号的关键参数挖掘方法定位参数子空间，并采用类型定制微调策略，实现模型遵循性与鲁棒性的均衡增强。
摘要：
检索增强生成（RAG）通过引入外部检索知识，为大型语言模型（LLMs）在幻觉生成和知识过时方面的问题提供了有效解决方案。然而，现有方法缺乏整合内部与外部知识的有效控制机制。受人类认知过程启发，我们提出了Parenting，一个新颖的框架，它解耦、识别并针对性地优化与遵循性和鲁棒性相关的参数子空间。具体而言，Parenting利用一种结合前向和反向传播信号的关键参数挖掘方法，来定位代表不同能力的子空间。然后，Parenting采用类型定制微调策略，对不同子空间应用特定且适当的优化，旨在实现遵循性和鲁棒性的均衡增强。在多种数据集和模型上的大量实验验证了我们方法的有效性和泛化能力。
英文摘要：
Retrieval-Augmented Generation (RAG) offers an effective solution to the issues faced by Large Language Models (LLMs) in hallucination generation and knowledge obsolescence by incorporating externally retrieved knowledge. However, existing methods lack effective control mechanisms for integrating internal and external knowledge. Inspired by human cognitive processes, we propose Parenting, a novel framework that decouples, identifies, and purposefully optimizes parameter subspaces related to adherence and robustness. Specifically, Parenting utilizes a key parameter mining method that combines forward and backward propagation signals to localize subspaces representing different capabilities. Then, Parenting employs a type-tailored tuning strategy, applying specific and appropriate optimizations to different subspaces, aiming to achieve a balanced enhancement of both adherence and robustness. Extensive experiments on various datasets and models validate the effectiveness and generalizability of our method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
