个性化大型视觉-语言模型
Personalized Large Vision-Language Models
AI总结：
PLVM通过预训练视觉编码器Aligner将指代概念与查询图像对齐，使LVLMs能在对话中持续低成本地添加新概念，实现更可定制且指代友好的个性化交互。
摘要：
个性化模型在图像生成领域获得了显著关注，但在大型视觉-语言模型（LVLMs）中仍研究不足。超越通用模型，通过个性化，LVLMs使用指代概念（例如，“Mike和Susan在谈话。”）而非通用形式（例如，“一个男孩和一个女孩在谈话。”）来处理交互式对话，使对话更加可定制且指代友好。此外，PLVM能够在对话过程中持续添加新概念而不产生额外成本，这显著增强了其实用性。PLVM提出了Aligner，一个预训练的视觉编码器，用于将指代概念与查询图像对齐。在对话过程中，它提取具有这些对应概念的参考图像的特征，并在查询图像中识别它们，从而实现个性化。我们注意到，Aligner的计算成本和参数量在整个框架中可以忽略不计。通过全面的定性和定量分析，我们揭示了PLVM的有效性和优越性。
英文摘要：
The personalization model has gained significant attention in image generation yet remains underexplored for large vision-language models (LVLMs). Beyond generic ones, with personalization, LVLMs handle interactive dialogues using referential concepts (e.g., ``Mike and Susan are talking.'') instead of the generic form (e.g., ``a boy and a girl are talking.''), making the conversation more customizable and referentially friendly. In addition, PLVM is equipped to continuously add new concepts during a dialogue without incurring additional costs, which significantly enhances the practicality. PLVM proposes Aligner, a pre-trained visual encoder to align referential concepts with the queried images. During the dialogues, it extracts features of reference images with these corresponding concepts and recognizes them in the queried image, enabling personalization. We note that the computational cost and parameter count of the Aligner are negligible within the entire framework. With comprehensive qualitative and quantitative analyses, we reveal the effectiveness and superiority of PLVM.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
