arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2508.09045cs.CV
收藏

逐查询视觉概念学习

Per-Query Visual Concept Learning

Bar‑Ilan University(巴伊兰大学);OriginAI;NVIDIA

Ori Malca, Dvir Samuel, Gal Chechik

AI总结:

提出一种特定于提示和噪声种子的个性化步骤,利用基于自注意力和交叉注意力的PDM特征损失项捕捉概念身份,在多种基础模型和个性化方法上显著提升了文本到图像个性化的语义相似度。

摘要:

视觉概念学习,又称文本到图像个性化,是向预训练模型教授新概念的过程。它具有从产品植入到娱乐和个性化设计的众多应用。在此我们表明,通过添加一个个性化步骤,许多现有方法可以得到大幅增强,该步骤(1)特定于提示和噪声种子,并且(2)使用基于自注意力和交叉注意力的两个损失项来捕捉个性化概念的身份。具体而言,我们利用PDM特征——先前设计用于捕捉身份——并展示如何用它们来提升个性化语义相似度。我们在六种不同的个性化方法和几种基础文本到图像模型(基于UNet和DiT)上评估了我们的方法所获得的收益。我们发现,即使与以前的逐查询个性化方法相比,也有显著改进。

英文摘要:

Visual concept learning, also known as Text-to-image personalization, is the process of teaching new concepts to a pretrained model. This has numerous applications from product placement to entertainment and personalized design. Here we show that many existing methods can be substantially augmented by adding a personalization step that is (1) specific to the prompt and noise seed, and (2) using two loss terms based on the self- and cross- attention, capturing the identity of the personalized concept. Specifically, we leverage PDM features -- previously designed to capture identity -- and show how they can be used to improve personalized semantic similarity. We evaluate the benefit that our method gains on top of six different personalization methods, and several base text-to-image models (both UNet- and DiT-based). We find significant improvements even over previous per-query personalization methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑