基于区域感知双模态直接偏好优化的组合式文本到图像生成
Compositional Text-to-Image Generation Via Region-aware Bimodal Direct Preference Optimization
AI总结：
提出BiDPO框架，通过构建大规模偏好数据集BiComp和扩展Diffusion DPO联合优化图像与文本偏好，结合区域级引导方法，提升文本到图像模型对复杂组合提示的生成保真度。
摘要：
尽管文本到图像（T2I）模型取得了快速进展，但生成准确反映复杂组合提示（涵盖属性绑定、对象关系、计数）的图像仍然具有挑战性。为了解决这个问题，我们提出了BiDPO，一个增强T2I模型组合式文本到图像生成能力的框架。我们首先引入一个精心设计的流程，构建大规模偏好数据集BiComp，并进行严格的质量控制。然后，我们将Diffusion DPO扩展到联合优化图像和文本偏好，这在提高模型遵循复杂文本提示生成方面被证明非常有效。为了进一步增强模型的细粒度对齐，我们采用区域级引导方法，聚焦于与组合概念相关的区域。实验结果表明，我们的BiDPO显著提高了组合保真度，在多个基准测试中持续优于先前方法。我们的方法突显了基于偏好微调在复杂文本到图像任务中的潜力，为现有技术提供了一种灵活且可扩展的替代方案。
英文摘要：
Despite the rapid progress of text-to-image (T2I) models, generating images that accurately reflect complex compositional prompts (covering attribute bindings, object relationships, counting) still remains challenging. To address this, we propose BiDPO, a framework to enhance T2I model's capability of compositional text-to-image generation. We begin by introducing an carefully designed pipeline to construct a large-scale preference dataset, BiComp, with strictly quality control. Then, we extend Diffusion DPO to jointly optimize image and text preferences, which is shown to greatly effective in improving the models to follow complex text prompt in generation. To further enhance the models for fine-grained alignment, we employ a region-level guidance method to focus on regions relevant to compositional concepts. Experimental results demonstrate that our BiDPO substantially improves compositional fidelity, consistently outperforming prior methods across multiple benchmarks. Our approach highlights the potential of preference-based fine-tuning for complex text-to-image tasks, offering a flexible and scalable alternative to existing techniques.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
