arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.12899cs.CV

用于改进少步文本到图像生成中指令遵循的轻量速度校正框架

A Lightweight Velocity Correction Framework for Improving Instruction Following in Few-Step Text-to-Image Generation

The University of Osaka(大阪大学);Alibaba Group(阿里巴巴集团);Peking University(北京大学);Huazhong University of Science and Technology(华中科技大学)

Yating Gao, Zhenlong Zhu, Wen Zihao, Lenan Zhang, Jiexiong Wang, Yang Yang, Guanjun Jiang, Yixiong Zou, Takuya Maekawa

AI总结:

该研究针对少步文本到图像生成器处理复杂提示时的错误,提出轻量速度残差校正框架,仅训练小参数校正器,在不增加前向传播次数的情况下,使指令遵循净胜率提升17.60个百分点。

摘要:

少步文本到图像生成器仅需少量网络前向传播即可生成高质量图像,但在处理包含多个主体、交互、空间关系、计数及属性绑定的复杂提示时表现不佳。能否在不改变生成器参数或增加前向传播次数的前提下减少这些错误?我们针对冻结的60亿参数生成器提出了一种轻量校正框架。该生成器会预测速度,速度决定其中间图像表示的更新方式;而参数少于2000万的校正器会在选定的采样步骤学习残差调整。由于早期校正会改变后续步骤遇到的状态,我们的轨迹流桥监督(On-Trajectory Flow-Bridge Supervision)会在校正生成过程中实际到达的状态上推导残差目标。单个校正器会在选定步骤间共享,并与生成器并行运行。仅需训练校正器,无需奖励优化或重新蒸馏。据我们所知,这是首个针对完全冻结的少步文本到图像生成器进行轻量速度残差适配的框架,它保留了生成器原有的采样计划和前向传播次数。在我们的目标复杂场景分布上,该框架在成对指令遵循评估中,相较于冻结的Z-Image-Turbo实现了17.60个百分点的净胜率提升,每幅图像仅增加26.9毫秒(0.84%)的端到端延迟,且骨干网络评估次数未增加。

英文摘要:

Few-step text-to-image generators produce high-quality images with few network forward passes, yet struggle with complex prompts involving multiple subjects, interactions, spatial relations, counting, and attribute binding. Can these errors be reduced without changing the generator's parameters or increasing its forward-pass count? We propose a lightweight correction framework for a frozen 6B-parameter generator. The generator predicts velocities that determine how its intermediate image representation is updated; a corrector with fewer than 20M parameters learns residual adjustments at selected sampling steps. Because earlier corrections change the states encountered at later steps, our On-Trajectory Flow-Bridge Supervision derives residual targets at the states actually reached during corrected generation. A single corrector is shared across selected steps and runs in parallel with the generator. Only the corrector is trained, requiring neither reward optimization nor re-distillation. To our knowledge, this is the first framework for lightweight velocity-residual adaptation of fully frozen few-step text-to-image generators that preserves their original sampling schedule and forward-pass count. On our target complex-scene distribution, the framework achieves a +17.60-percentage-point NetWin over frozen Z-Image-Turbo in pairwise instruction-following evaluation, with only 26.9 ms (0.84%) additional end-to-end latency per image and no increase in backbone evaluations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑