Diffuse to Choose：在用于 Virtual Try-All 的潜空间扩散模型中丰富图像条件修复能力
Diffuse to Choose: Enriching Image Conditioned Inpainting in Latent Diffusion Models for Virtual Try-All
AI总结：
本文提出 Diffuse to Choose，通过向主扩散模型注入参考图像细粒度特征并加入感知损失，实现兼顾速度、细节保真与语义操控的 Virtual Try-All 图像修复。
摘要：
随着在线购物不断增长，买家能够在自身场景中虚拟可视化商品——我们将这一现象定义为“Virtual Try-All”——已变得至关重要。近期的扩散模型天然包含一个世界模型，因此适合在修复（inpainting）语境下完成该任务。然而，传统的图像条件扩散模型往往无法捕捉商品的细粒度细节。相比之下，DreamPaint 等个性化驱动模型擅长保留物品细节，但其并未针对实时应用进行优化。我们提出“Diffuse to Choose”，这是一种新颖的基于扩散的图像条件修复模型，能够在快速推理与保留给定参考物品的高保真细节之间高效取得平衡，同时确保对给定场景内容进行准确的语义操控。我们的方法基于将参考图像中的细粒度特征直接注入主扩散模型的潜空间特征图，并结合感知损失以进一步保留参考物品的细节。我们在内部数据集和公开可用数据集上进行了广泛测试，结果表明 Diffuse to Choose 优于现有的零样本扩散修复方法，以及 DreamPaint 等少样本扩散个性化算法。
英文摘要：
As online shopping is growing, the ability for buyers to virtually visualize products in their settings-a phenomenon we define as "Virtual Try-All"-has become crucial. Recent diffusion models inherently contain a world model, rendering them suitable for this task within an inpainting context. However, traditional image-conditioned diffusion models often fail to capture the fine-grained details of products. In contrast, personalization-driven models such as DreamPaint are good at preserving the item's details but they are not optimized for real-time applications. We present "Diffuse to Choose," a novel diffusion-based image-conditioned inpainting model that efficiently balances fast inference with the retention of high-fidelity details in a given reference item while ensuring accurate semantic manipulations in the given scene content. Our approach is based on incorporating fine-grained features from the reference image directly into the latent feature maps of the main diffusion model, alongside with a perceptual loss to further preserve the reference item's details. We conduct extensive testing on both in-house and publicly available datasets, and show that Diffuse to Choose is superior to existing zero-shot diffusion inpainting methods as well as few-shot diffusion personalization algorithms like DreamPaint.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
