DRC：通过解耦表示组合增强个性化图像生成
DRC: Enhancing Personalized Image Generation via Disentangled Representation Composition
AI总结：
DRC通过解耦表示组合框架，分别提取用户风格与语义意图，解决LMMs中的引导崩溃问题，实现可控且有效的个性化图像生成。
摘要：
个性化图像生成已成为多模态内容创作中的一个有前景的方向。它旨在通过利用用户交互历史图像和多模态指令，合成符合个人风格偏好（如配色方案、角色外观、布局）和语义意图（如情感、动作、场景上下文）的图像。尽管取得了显著进展，现有方法——无论是基于扩散模型、大语言模型还是大型多模态模型（LMMs）——在准确捕获和融合用户风格偏好与语义意图方面仍存在困难。特别是，最先进的基于LMM的方法受困于视觉特征的纠缠，导致引导崩溃（Guidance Collapse），即生成的图像无法保留用户偏好的风格或反映指定的语义。为了解决这些局限性，我们引入了DRC，一种新颖的个性化图像生成框架，通过解耦表示组合（Disentangled Representation Composition）增强LMMs。DRC分别从历史图像和参考图像中显式提取用户风格偏好和语义意图，形成用户特定的潜在指令，以指导LMMs内的图像生成。具体而言，它涉及两个关键的学习阶段：1）解耦学习，采用双塔解耦器显式分离风格和语义特征，并通过带有难度感知重要性采样的重建驱动范式进行优化；2）个性化建模，应用语义保持增强以有效适应解耦表示，实现鲁棒的个性化生成。在两个基准上的大量实验表明，DRC在有效缓解引导崩溃问题的同时展示了有竞争力的性能，强调了解耦表示学习对于可控且有效的个性化图像生成的重要性。
英文摘要：
Personalized image generation has emerged as a promising direction in multimodal content creation. It aims to synthesize images tailored to individual style preferences (e.g., color schemes, character appearances, layout) and semantic intentions (e.g., emotion, action, scene contexts) by leveraging user-interacted history images and multimodal instructions. Despite notable progress, existing methods -- whether based on diffusion models, large language models, or Large Multimodal Models (LMMs) -- struggle to accurately capture and fuse user style preferences and semantic intentions. In particular, the state-of-the-art LMM-based method suffers from the entanglement of visual features, leading to Guidance Collapse, where the generated images fail to preserve user-preferred styles or reflect the specified semantics. To address these limitations, we introduce DRC, a novel personalized image generation framework that enhances LMMs through Disentangled Representation Composition. DRC explicitly extracts user style preferences and semantic intentions from history images and the reference image, respectively, to form user-specific latent instructions that guide image generation within LMMs. Specifically, it involves two critical learning stages: 1) Disentanglement learning, which employs a dual-tower disentangler to explicitly separate style and semantic features, optimized via a reconstruction-driven paradigm with difficulty-aware importance sampling; and 2) Personalized modeling, which applies semantic-preserving augmentations to effectively adapt the disentangled representations for robust personalized generation. Extensive experiments on two benchmarks demonstrate that DRC shows competitive performance while effectively mitigating the guidance collapse issue, underscoring the importance of disentangled representation learning for controllable and effective personalized image generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
