门与融合：面向视觉语言模型的零样本组合个性化
Gate-and-Merge: Zero-shot Compositional Personalization of Vision Language Models
AI总结：
本文提出Gate-and-Merge框架，实现视觉语言模型的零样本组合个性化，通过轻量LoRA适配器和门控机制，在无需共现训练的情况下提升多概念联合识别性能。
摘要：
本文针对视觉语言模型（VLMs）的组合个性化问题，提出Gate-and-Merge框架。该框架允许在无需共现训练的情况下实现组合个性化。在个性化过程中，每个概念被独立学习为轻量LoRA适配器，并配以概念标记。基础模型保持不变，概念保持解耦。在推理阶段，通过在权重空间中直接融合概念特定的LoRA更新来实现组合。为抑制无关激活并防止干扰，采用门控机制来估计文本和视觉线索，并仅选择对预测有贡献的模块。进一步通过结合最相关且互为一致的更新来稳定组合，有助于保持每个概念的身份。定量和定性分析显示，在单概念和组合设置下的多个个性化任务中均实现了性能提升。
英文摘要：
This paper tackles compositional personalization of vision-language models (VLMs). In this problem, multiple user-defined concepts must be recognized or described jointly at test time. We introduce Gate-and-Merge, a zero-shot framework that enables compositional personalization without the need for co-occurrence training. During personalization, each concept is learned independently as a lightweight LoRA adapter, paired with a concept token. The base model remains unchanged and concepts are kept disentangled. At inference, we enable composition by merging concept-specific LoRA updates directly in weight space. To suppress irrelevant activations and prevent interference, a gating mechanism is employed to estimate textual and visual cues and select only the modules that contribute to the prediction. We further stabilize composition by combining only the most meaningful and mutually consistent updates, helping preserve each concept's identity. Our quantitative and qualitative analyses show consistent gains in performance across multiple personalization tasks in both single-concept and compositional settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
