通过潜空间拼接和掩码条件流匹配驯服扩散模型中的身份一致性与提示多样性
Taming Identity Consistency and Prompt Diversity in Diffusion Models via Latent Concatenation and Masked Conditional Flow Matching
AI总结：
针对主体驱动图像生成中身份一致性与提示多样性的权衡问题，提出潜空间拼接策略结合掩码CFM目标的LoRA微调扩散模型，辅以两阶段蒸馏数据整理框架与CHARIS评估框架，实现鲁棒身份保留与生成能力扩展。
摘要：
主体驱动图像生成旨在在多样语境下合成特定主体的新颖描绘，同时保留其核心身份特征。实现强身份一致性与高提示多样性存在根本权衡。我们提出一种采用潜空间拼接策略的LoRA微调扩散模型，联合处理参考图像与目标图像，并结合掩码条件流匹配（CFM）目标。该方法无需修改架构即可实现鲁棒的身份保留。为促进大规模训练，我们引入两阶段蒸馏数据整理框架：第一阶段利用数据恢复和基于VLM的过滤，从多样源创建紧凑高质量种子数据集；第二阶段利用这些整理样本进行参数高效微调，从而扩展跨不同主体和语境的生成能力。最后，为过滤和质量评估，我们提出CHARIS——一个细粒度评估框架，沿身份一致性、提示遵循度、区域颜色保真度、视觉质量和变换多样性五个关键维度进行属性级比较。
英文摘要：
Subject-driven image generation aims to synthesize novel depictions of a specific subject across diverse contexts while preserving its core identity features. Achieving both strong identity consistency and high prompt diversity presents a fundamental trade-off. We propose a LoRA fine-tuned diffusion model employing a latent concatenation strategy, which jointly processes reference and target images, combined with a masked Conditional Flow Matching (CFM) objective. This approach enables robust identity preservation without architectural modifications. To facilitate large-scale training, we introduce a two-stage Distilled Data Curation Framework: the first stage leverages data restoration and VLM-based filtering to create a compact, high-quality seed dataset from diverse sources; the second stage utilizes these curated examples for parameter-efficient fine-tuning, thus scaling the generation capability across various subjects and contexts. Finally, for filtering and quality assessment, we present CHARIS, a fine-grained evaluation framework that performs attribute-level comparisons along five key axes: identity consistency, prompt adherence, region-wise color fidelity, visual quality, and transformation diversity.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
