arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2606.27978cs.CVcs.AI

像素空间自回归图像生成的并行展开近似

Parallel Rollout Approximation for Pixel-Space Autoregressive Image Generation

Peking University(北京大学);DP Technology

Jiayi Xu, Di He, Guolin Ke

AI总结:

提出并行展开近似(PRA)框架,通过生成低维中间状态并映射回像素令牌,解决像素空间连续令牌自回归生成中高维补丁误差大和训练-推理差距问题,在ImageNet-1K上以135M参数实现FID 2.58,511M参数达到1.94,创像素空间AR模型新纪录。

摘要:

像素空间连续令牌自回归(AR)生成直接将图像建模为原始像素补丁的序列,避免了离散令牌化或单独预训练的令牌化器。然而,它面临耦合挑战:高维补丁生成导致大的单步误差,教师强制训练造成训练-推理差距,使这些误差在AR步骤中累积。现有的修复方法如$x$-预测和输入噪声注入只能部分缓解这些问题。精确的展开训练更好地匹配推理时的条件,但由于极其缓慢的序列采样而不实用。我们提出\emph{并行展开近似}(PRA),一个可扩展的框架,联合解决这两个挑战。PRA生成低维中间状态而不是高维像素补丁,然后通过像素解码器将它们映射回像素空间令牌,保持像素输入、像素输出的AR接口。它还通过相同的中间状态到像素路径(在推理时使用)独立地在各个位置构建类似推理的像素输入,近似推理时展开期间遇到的像素反馈接口,同时保留并行的教师强制训练。在类别条件ImageNet-1K生成任务中,分辨率为$256\ imes256$,PRA-S(135M参数)实现了2.58的FID,超过了之前十亿级像素空间AR结果3.60。扩展到PRA-L(511M参数)进一步将FID提升至1.94,在像素空间AR模型中建立了新的最先进水平。除了生成,PRA在ImageNet分类探测准确率上优于其他AR和扩散基线,表明其在统一像素空间图像生成和理解方面的潜力。

英文摘要:

Pixel-space continuous-token autoregressive (AR) generation directly models images as sequences of raw pixel patches, avoiding discrete tokenization or a separately pretrained tokenizer. However, it faces coupled challenges: high-dimensional patch generation causes large single-step errors, and teacher-forced training creates a train--inference gap that makes these errors accumulate across AR steps. Existing fixes such as $x$-prediction and input noise injection only partially mitigate these issues. Exact rollout training better matches inference-time conditions, but is impractical due to prohibitively slow sequential sampling. We propose \emph{Parallel Rollout Approximation} (PRA), a scalable framework that addresses both challenges jointly. PRA generates low-dimensional intermediate states instead of high-dimensional pixel patches, then maps them back to pixel-space tokens with a pixel decoder, preserving a pixel-in, pixel-out AR interface. It also constructs inference-like pixel inputs through the same intermediate-state-to-pixel path used at inference, independently across positions, approximating the pixel-feedback interface encountered during inference-time rollout while retaining parallel teacher-forced training. On class-conditional ImageNet-1K generation at $256\times256$ resolution, PRA-S with 135M parameters achieves an FID of 2.58, surpassing the previous billion-scale pixel-space AR result of 3.60. Scaling to PRA-L with 511M parameters further improves FID to 1.94, establishing a new state of the art among pixel-space AR models. Beyond generation, PRA achieves higher ImageNet classification probing accuracy than other AR and diffusion baselines, suggesting its potential for unified pixel-space image generation and understanding.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑