LoRA-Loop：为持续视觉语言模型学习闭合合成重放循环
LoRA-Loop: Closing the Synthetic Replay Cycle for Continual VLM Learning
AI总结：
针对视觉语言模型持续学习中合成重放样本与下游任务对齐偏差的问题，提出LoRA增强的两阶段置信度筛选合成重放框架，在MTIL基准上实现可塑性、稳定性与零样本能力的最优平衡。
摘要：
视觉语言模型的持续学习已通过合成重放取得了显著性能，该方法利用Stable Diffusion（稳定扩散模型）生成样本，在微调过程中起到正则化作用并保留知识。然而，现实世界的下游应用往往存在生成器无法捕捉的领域特定细微差异和细粒度语义，导致合成重放方法生成的样本存在对齐偏差，会误导微调过程并削弱对先前知识的保留。在本研究中，我们提出一种LoRA增强的合成重放框架，将任务特定的低秩适配器注入冻结的Stable Diffusion模型中，高效捕捉每个新任务独特的视觉和语义模式。具体而言，我们引入了基于置信度的两阶段样本选择机制：首先根据微调后VLM的置信度对真实任务数据排序，让LoRA微调聚焦于最具代表性的样本；随后生成合成样本，并再次通过置信度筛选样本用于蒸馏。我们的方法可与现有重放流水线无缝集成——只需替换为适配后的生成器即可提升重放保真度。在Multi-domain Task Incremental Learning（MTIL，多领域任务增量学习）基准上的大量实验表明，我们的方法优于以往的合成重放技术，在可塑性、稳定性和零样本能力之间实现了最优平衡。这些结果证明了通过LoRA进行生成器适配对于实现视觉语言模型鲁棒持续学习的有效性。
英文摘要：
Continual learning for vision-language models has achieved remarkable performance through synthetic replay, where samples are generated using Stable Diffusion to regularize during finetuning and retain knowledge. However, real-world downstream applications often exhibit domain-specific nuances and fine-grained semantics not captured by generators, causing synthetic-replay methods to produce misaligned samples that misguide finetuning and undermine retention of prior knowledge. In this work, we propose a LoRA-enhanced synthetic-replay framework that injects task-specific low-rank adapters into a frozen Stable Diffusion model, efficiently capturing each new task's unique visual and semantic patterns. Specifically, we introduce a two-stage, confidence-based sample selection: we first rank real task data by post-finetuning VLM confidence to focus LoRA finetuning on the most representative examples, then generate synthetic samples and again select them by confidence for distillation. Our approach integrates seamlessly with existing replay pipelines-simply swap in the adapted generator to boost replay fidelity. Extensive experiments on the Multi-domain Task Incremental Learning (MTIL) benchmark show that our method outperforms previous synthetic-replay techniques, achieving an optimal balance among plasticity, stability, and zero-shot capability. These results demonstrate the effectiveness of generator adaptation via LoRA for robust continual learning in VLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
