arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.00483cs.CVcs.AI

RAISE:基于需求的进化精炼用于无训练文本到图像对齐

RAISE: Requirement-Adaptive Evolutionary Refinement for Training-Free Text-to-Image Alignment

Department of ECE, University of Alberta, Canada(阿尔伯塔大学电子工程系);Huawei Technologies, Canada(华为技术有限公司)

Liyao Jiang, Ruichen Chen, Chao Gao, Di Niu

AI总结:

RAISE是一种无训练、需求驱动的进化框架,通过动态调整生成过程实现高效且通用的文本到图像对齐。

摘要:

最近的文本到图像(T2I)扩散模型实现了显著的现实感,但忠实的提示-图像对齐仍然具有挑战性,尤其是在涉及多个对象、关系和细粒度属性的复杂提示中。现有的无训练推理时间缩放方法依赖于固定的迭代预算,无法适应提示难度,而反射调节模型需要精心编纂的反射数据集和扩散和视觉语言模型的大量联合微调,通常会过度拟合反射路径数据,并且在不同模型之间缺乏可转移性。我们介绍了RAISE(需求自适应自我改进进化),一种无训练、需求驱动的进化框架,用于自适应T2I生成。RAISE将图像生成公式化为一个需求驱动的自适应缩放过程,在推理时通过一组多样化的精炼动作(包括提示重写、噪声重采样和指导性编辑)进化候选人群体。每个生成都经过结构化要求检查表的验证,使系统能够动态识别未满足的项目,并只在需要的地方分配进一步的计算。这实现了自适应的测试时间缩放,使计算努力与语义查询复杂性相匹配。在GenEval和DrawBench上,RAISE实现了最先进的对齐(整体GenEval得分为0.94),同时生成的样本数量(减少30-40%)和VLM调用(减少80%)比先前的缩放和反射调节基线更少,展示了高效、可推广且模型无关的多轮自我改进。代码可在https://github.com/LiyaoJiang1998/RAISE上获得。

英文摘要:

Recent text-to-image (T2I) diffusion models achieve remarkable realism, yet faithful prompt-image alignment remains challenging, particularly for complex prompts with multiple objects, relations, and fine-grained attributes. Existing training-free inference-time scaling methods rely on fixed iteration budgets that cannot adapt to prompt difficulty, while reflection-tuned models require carefully curated reflection datasets and extensive joint fine-tuning of diffusion and vision-language models, often overfitting to reflection paths data and lacking transferability across models. We introduce RAISE (Requirement-Adaptive Self-Improving Evolution), a training-free, requirement-driven evolutionary framework for adaptive T2I generation. RAISE formulates image generation as a requirement-driven adaptive scaling process, evolving a population of candidates at inference time through a diverse set of refinement actions-including prompt rewriting, noise resampling, and instructional editing. Each generation is verified against a structured checklist of requirements, enabling the system to dynamically identify unsatisfied items and allocate further computation only where needed. This achieves adaptive test-time scaling that aligns computational effort with semantic query complexity. On GenEval and DrawBench, RAISE attains state-of-the-art alignment (0.94 overall GenEval) while incurring fewer generated samples (reduced by 30-40%) and VLM calls (reduced by 80%) than prior scaling and reflection-tuned baselines, demonstrating efficient, generalizable, and model-agnostic multi-round self-improvement. Code is available at https://github.com/LiyaoJiang1998/RAISE.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑