异质图像图神经网络：用于图像合成的图条件扩散模型
Heterogeneous Image GNN: Graph-Conditioned Diffusion for Image Synthesis
AI总结：
本文提出异质图像图（HIG）表示和保幅图神经网络，将异构图条件集成到EDM2扩散模型中，在COCO-stuff和Visual Genome上超越现有方法，实现基于图属性与关系的图像合成条件控制。
摘要：
我们提出了一种利用异构图数据对基于扩散的图像合成模型进行条件控制的新方法。现有方法通常将条件变量直接整合进模型架构，要么通过关注文本潜在表示的交叉注意力层，要么通过空间上限制生成的图像拼接来实现。然而，这些方法难以处理涉及多样化、关系型条件变量的复杂场景，这类变量更自然地表示为非结构化图。本文提出了异质图像图（HIG），这是一种新颖的表示方法，将条件变量和目标图像建模为两个相互连接的图，从而能够高效处理变长条件输入及其关系。我们还提出了一种保幅图神经网络，采用ControlNet方法将HIG集成到现有的EDM2扩散模型中。我们的方法在COCO-stuff和Visual Genome数据集上的多种条件输入上优于当前最优方法，并展示了基于HIG中边所表示的图属性和关系进行条件控制的能力。
英文摘要：
We introduce a novel method for conditioning diffusion-based image synthesis models with heterogeneous graph data. Existing approaches typically incorporate conditioning variables directly into model architectures, either through cross-attention layers that attend to text latents or image concatenation that spatially restrict generation. However, these methods struggle to handle complex scenarios involving diverse, relational conditioning variables, which are more naturally represented as unstructured graphs. This paper presents Heterogeneous Image Graphs (HIG), a novel representation that models conditioning variables and target images as two interconnected graphs, enabling efficient handling of variable-length conditioning inputs and their relationships. We also propose a magnitude-preserving GNN that integrates the HIG into the existing EDM2 diffusion model using a ControlNet approach. Our approach improves upon the SOTA on a variety of conditioning inputs for the COCO-stuff and Visual Genome datasets, and showcases the ability to condition on graph attributes and relationships represented by edges in the HIG.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
