生成增强生成：一种用于大语言模型私有知识注入的即插即用框架
Generation-Augmented Generation: A Plug-and-Play Framework for Private Knowledge Injection in Large Language Models
AI总结：
本文提出GAG框架，通过将私有专业知识作为辅助模态注入冻结的基础模型，解决私有知识注入中的效率与效果平衡问题，在科学和通用领域问答中均表现出色。
摘要：
在材料科学、生物医学和金融等领域，大语言模型（LLMs）的高风险部署需要注入专有、领域特定且在公共预训练中代表性不足的私有知识。然而，私有知识注入的两种主流范式各有缺陷：微调在持续更新时成本高且易导致灾难性遗忘和一般能力退化；检索增强生成（RAG）保持基础模型完整但因片段化证据、检索不匹配和长上下文压力而脆弱。受多模态LLMs对异构模态对齐共享语义空间的启发，本文提出生成增强生成（GAG），将私有专业知识视为辅助模态，并通过紧凑且预算恒定的潜在接口注入冻结的基础模型。具体而言，GAG从轻量级领域专家中蒸馏问题条件的专家知识到多槽潜在记忆中，通过每槽跨层融合整合多层专家信号，并通过门控残差投影对齐到冻结的基础模型，同时支持可扩展的混合领域部署并提供可靠的选性激活。在统一的混合领域评估中，GAG在两个科学私有领域问答基准（催化材料和免疫学佐剂）以及通用领域查询上均优于强检索基和参数高效微调基线，在专家问答中表现优异，同时保持通用领域能力，实现高度可靠的路由，并提供有利的效率-效果权衡。代码和数据集在补充材料中提供。代码可在https://github.com/360CVGroup/GAG公开获取。
英文摘要：
In domains such as materials science, biomedicine, and finance, high-stakes deployment of large language models (LLMs) requires injecting private, domain-specific knowledge that is proprietary, fast-evolving, and under-represented in public pretraining. However, the two dominant paradigms for private knowledge injection each have clear drawbacks: fine-tuning is expensive to iterate under continual updates that can induce catastrophic forgetting and general-capability regression; retrieval-augmented generation (RAG) keeps the base model intact but remains brittle in specialized private corpora due to chunk-induced evidence fragmentation, retrieval mismatch, and long-context pressure. Inspired by how multimodal LLMs align heterogeneous modalities into a shared semantic space, we propose Generation-Augmented Generation (GAG), which treats private expertise as an auxiliary modality and injects it into a frozen base model through a compact, constant-budget latent interface. Concretely, GAG distills question-conditioned specialist knowledge from lightweight domain experts into multi-slot latent memories, integrates multi-layer expert signals via per-slot cross-layer fusion, and aligns them to the frozen base model through gated residual projection, while supporting scalable mixed-domain deployment with reliable selective activation. In a unified mixed-domain evaluation spanning two scientific private-domain QA benchmarks (catalytic materials and immunology adjuvant) together with general-domain queries, GAG consistently outperforms strong retrieval-based and parameter-efficient fine-tuning baselines on specialist QA, while preserving general-domain capability, achieving highly reliable routing, and offering a favorable efficiency--effectiveness trade-off. Code and datasets are provided in the supplementary material. Code is publicly available at https://github.com/360CVGroup/GAG.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
