arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2511.19965cs.CV
收藏

HiCoGen: 通过强化学习在扩散模型中实现层次化组合文本到图像生成

HiCoGen: Hierarchical Compositional Text-to-Image Generation in Diffusion Models via Reinforcement Learning

SKL-IOTSC, CIS, University of Macau(澳门大学SKL-IOTSC、CIS);Zhejiang ZEEKR Automobile Research & Development Co., Ltd.(浙江浙汽汽车研发有限公司)

Hongji Yang, Yucheng Zhou, Wencheng Han, Runzhou Tao, Zhongying Qiu, Jianfei Yang, Jianbing Shen

AI总结:

HiCoGen通过强化学习和层次化合成框架提升文本到图像生成的组合性和准确性。

摘要:

最近的扩散模型进展展示了在简单提示下生成高质量图像的 impressive 能力。然而,当面对涉及多个物体和层次结构的复杂提示时,现有模型难以准确遵循指令,导致概念遗漏、混淆和组合性差的问题。为了解决这些限制,我们提出了一种基于新型合成链(CoS)范式的层次化组合生成框架(HiCoGen)。不同于单一生成,HiCoGen首先利用大型语言模型(LLM)将复杂提示分解为最小的语义单元。然后,它迭代合成这些单元,其中每一步生成的图像为下一步提供关键的视觉上下文,确保所有文本概念都被忠实构建到最终场景中。为进一步优化这一过程,我们引入了强化学习(RL)框架。关键的是,我们发现标准扩散采样器的有限探索阻碍了有效的 RL。我们理论证明,通过在早期生成阶段集中随机性可最大化样本多样性,并基于这一见解,提出了一种新的衰减随机性调度方案以增强探索。我们的 RL 算法由一种分层奖励机制引导,该机制共同评估图像的全局、主体和关系层面。我们还构建了 HiCoPrompt,一个具有层次化提示的新文本到图像基准,以进行严格评估。实验显示,我们的方法在概念覆盖和组合性准确性方面显著优于现有方法。

英文摘要:

Recent advances in diffusion models have demonstrated impressive capability in generating high-quality images for simple prompts. However, when confronted with complex prompts involving multiple objects and hierarchical structures, existing models struggle to accurately follow instructions, leading to issues such as concept omission, confusion, and poor compositionality. To address these limitations, we propose a Hierarchical Compositional Generative framework (HiCoGen) built upon a novel Chain of Synthesis (CoS) paradigm. Instead of monolithic generation, HiCoGen first leverages a Large Language Model (LLM) to decompose complex prompts into minimal semantic units. It then synthesizes these units iteratively, where the image generated in each step provides crucial visual context for the next, ensuring all textual concepts are faithfully constructed into the final scene. To further optimize this process, we introduce a reinforcement learning (RL) framework. Crucially, we identify that the limited exploration of standard diffusion samplers hinders effective RL. We theoretically prove that sample diversity is maximized by concentrating stochasticity in the early generation stages and, based on this insight, propose a novel Decaying Stochasticity Schedule to enhance exploration. Our RL algorithm is then guided by a hierarchical reward mechanism that jointly evaluates the image at the global, subject, and relationship levels. We also construct HiCoPrompt, a new text-to-image benchmark with hierarchical prompts for rigorous evaluation. Experiments show our approach significantly outperforms existing methods in both concept coverage and compositional accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑