分块进展:用于自回归图像生成的测试时扩展
Progress by Pieces: Test-Time Scaling for Autoregressive Image Generation
AI总结:
GridAR通过网格分区逐步生成和布局指定提示重述策略,在有限测试时扩展下提升视觉自回归模型的生成质量与编辑效果。
摘要:
最近的视觉自回归(AR)模型在文本到图像生成中展示了有前途的能力,其操作方式类似于大型语言模型。尽管测试时计算扩展在使自然语言任务产生推理增强的输出方面取得了显著成功,但将其适应到视觉AR模型仍处于未探索的阶段,并带来了独特的挑战。简单地应用测试时扩展策略,如Best-of-N,可能效果不佳:它们会消耗完整长度的计算来处理错误的生成轨迹,而逐行扫描解码方案缺乏整个画布的蓝图,限制了扩展的好处,因为只有少数与提示对齐的候选生成。为了解决这些问题,我们引入了GridAR,一个测试时扩展框架,旨在从视觉AR模型中提取最佳可能的结果。GridAR采用了一种网格分区的逐步生成方案,在画布上为同一位置生成多个部分候选,在不可行的情况下进行早期修剪,并将可行的候选固定为锚点以指导后续解码。此外,我们还提出了一种布局指定的提示重述策略,该策略检查部分视图以推断一个可行的布局以满足提示。重述后的提示然后指导后续的图像生成以缓解蓝图不足的问题。共同作用下,GridAR在有限的测试时扩展下实现了更高质量的结果:在T2I-CompBench++上,当N=4时,它甚至比Best-of-N(N=8)高出14.4%,同时成本降低了25.6%。它还扩展到了自回归图像编辑,显示出与之相当的编辑质量和在PIE-Bench上比更大N基线高出13.9%的语义保留收益。
英文摘要:
Recent visual autoregressive (AR) models have shown promising capabilities in text-to-image generation, operating in a manner similar to large language models. While test-time computation scaling has brought remarkable success in enabling reasoning-enhanced outputs for challenging natural language tasks, its adaptation to visual AR models remains unexplored and poses unique challenges. Naively applying test-time scaling strategies such as Best-of-N can be suboptimal: they consume full-length computation on erroneous generation trajectories, while the raster-scan decoding scheme lacks a blueprint of the entire canvas, limiting scaling benefits as only a few prompt-aligned candidates are generated. To address these, we introduce GridAR, a test-time scaling framework designed to elicit the best possible results from visual AR models. GridAR employs a grid-partitioned progressive generation scheme in which multiple partial candidates for the same position are generated within a canvas, infeasible ones are pruned early, and viable ones are fixed as anchors to guide subsequent decoding. Coupled with this, we present a layout-specified prompt reformulation strategy that inspects partial views to infer a feasible layout for satisfying the prompt. The reformulated prompt then guides subsequent image generation to mitigate the blueprint deficiency. Together, GridAR achieves higher-quality results under limited test-time scaling: with N=4, it even outperforms Best-of-N (N=8) by 14.4% on T2I-CompBench++ while reducing cost by 25.6%. It also generalizes to autoregressive image editing, showing comparable edit quality and a 13.9% gain in semantic preservation on PIE-Bench over larger-N baselines.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。