Recent advances in diffusion models have demonstrated impressive capability in generating high-quality images for simple prompts. However, when confronted with complex prompts involving multiple objects and hierarchical structures, existing models struggle to accurately follow instructions, leading to issues such as concept omission, confusion, and poor compositionality. To address these limitations, we propose a Hierarchical Compositional Generative framework (HiCoGen) built upon a novel Chain of Synthesis (CoS) paradigm. Instead of monolithic generation, HiCoGen first leverages a Large Language Model (LLM) to decompose complex prompts into minimal semantic units. It then synthesizes these units iteratively, where the image generated in each step provides crucial visual context for the next, ensuring all textual concepts are faithfully constructed into the final scene. To further optimize this process, we introduce a reinforcement learning (RL) framework. Crucially, we identify that the limited exploration of standard diffusion samplers hinders effective RL. We theoretically prove that sample diversity is maximized by concentrating stochasticity in the early generation stages and, based on this insight, propose a novel Decaying Stochasticity Schedule to enhance exploration. Our RL algorithm is then guided by a hierarchical reward mechanism that jointly evaluates the image at the global, subject, and relationship levels. We also construct HiCoPrompt, a new text-to-image benchmark with hierarchical prompts for rigorous evaluation. Experiments show our approach significantly outperforms existing methods in both concept coverage and compositional accuracy.
MCCD: Multi-Agent Collaboration-based Compositional Diffusion for Complex Text-to-Image Generation
Academy for Engineering and Technology(工程与技术学院);Cognition and Intelligent Technology Laboratory (CIT Lab)(认知与智能技术实验室);School of Future Science and Engineering(未来科学与工程学院);Jilin Provincial Key Laboratory of Intelligence Science and Engineering(吉林省智能科学与工程重点实验室);Engineering Research Center of AI and Robotics, Ministry of Education(人工智能与机器人工程研究中心)
Hierarchical Compositional Representations for Few-shot Action Recognition
Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CAS(中国科学院智能信息处理重点实验室,中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Hangzhou Institute for Advanced Study, UCAS(国科大杭州高等研究院);Beijing Huawei Cloud Computing Technologies Co., Ltd(北京华为云计算技术有限公司)
University of California, Los Angeles(加利福尼亚大学洛杉矶分校);Tensor Vision Technologies(张量视觉技术公司)
视觉对象是由一个递归层次的知觉整体和部分,其属性,如形状,反射率和颜色,构成了一个层次的内在因果因素的对象外观。然而,物体外观是物体的内在和外在因果因素的组成结果,其中外在因果因素与照明和成像条件有关。因此,本文提出了一个统一的张量模型的整体和部分,并介绍了一个组成层次张量分解,解开层次因果结构的对象图像形成,并纳入多线性块张量分解作为一个特殊情况。所得到的对象表示是整体和部分表示的可解释的组合选择,其使得对象识别对遮挡鲁棒并且减少训练数据要求。我们通过在一个极其简化的合成图像数据集上进行训练,在人脸识别的背景下演示了我们的方法,并在两个数据集上报告了人脸验证结果-Freiburg数据集和由真实世界图像组成的Labeled Face in the Wild(LFW)数据集,从而证实了我们的方法适用于数据匮乏的领域。