发表机构
School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机与信息工程学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对扩散模型训练的常识偏差致罕见概念图像生成不佳的问题，提出基于反事实推理的CI-Diff方法，阻断干扰，利用自然直接效应解耦属性，重新设计引导机制，经实验验证该方法优于现有模型。
AI 中文摘要
罕见概念生成专注于根据描述具有异常属性对象的文本提示合成定制图像。以往工作生成的图像与罕见概念不一致。我们发现这源于扩散模型训练阶段的常识偏差。为此，我们提出基于反事实推理的扩散方法CI-Diff，它能阻断常识偏差干扰，利用自然直接效应捕捉罕见概念文本提示对图像生成的独立影响，解耦罕见概念与异常属性。我们还重新设计无分类器引导机制突出非典型属性。实验验证了CI-Diff优于现有模型。
英文摘要
Rare concept generation focuses on synthesizing customized images conditioned on text prompts that describe objects with unusual attributes. Previous works failed to align the generated images with rare concepts, resulting in incorrect attribute rendering or inconsistent composition of concepts. Such failures, as we observed, stem from the inherent common knowledge bias in the training stage of diffusion models, where objects are strongly associated with their common attributes, making it difficult to break these associations when generating rare concepts. To address such challenges, in this paper, we propose a novel Counterfactual Inference-based Diffusion approach, dubbed CI-Diff. CI-Diff blocks the interference of the model's inherent common knowledge bias and utilizes the Natural Direct Effect to capture the independent influence of the text prompt of rare concepts on image generation so that decoupling the unusual attributes from the rare concepts. To this end, we reformulate the classifier-free guidance mechanism to highlight the atypical attributes. To the best of our knowledge, we are the first to introduce causal inference into the rare concept generation task. Extensive experiments on the RareBench benchmark validate the superiority of CI-Diff over state-of-the-art diffusion models. Our code can be accessed from https://github.com/200204jzy/CI-Diff.
CommentsComments: 17 pages, 15 figures, to appear at ACM Multimedia 2026
