具有随机并行解码的自回归图像生成
Autoregressive Image Generation with Randomized Parallel Decoding
AI总结：
ARPG通过解耦的位置指导与内容表示，实现高效的随机顺序图像生成，提升推理速度和零样本泛化能力。
摘要：
我们引入了ARPG，一种新型的视觉自回归模型，能够实现随机并行生成，解决传统光栅顺序方法固有的局限性，这些方法由于其顺序且预定义的标记生成顺序，阻碍了推理效率和零样本泛化能力。我们的关键见解是，有效的随机顺序建模需要明确的指导来确定下一个预测标记的位置。为此，我们提出了一种新的解耦解码框架，将位置指导与内容表示解耦，分别将其编码为查询和键值对。通过将此指导直接纳入因果注意力机制，我们的方法实现了完全随机顺序的训练和生成，消除了对双向注意力的需求。因此，ARPG可以轻松泛化到零样本任务，如图像修复、外补和分辨率扩展。此外，它通过同时处理多个查询使用共享的KV缓存来支持并行推理。在ImageNet-1K 256基准测试中，我们的方法仅用32次采样步骤就达到了FID为1.83的性能，相比类似规模的代表性自回归模型，在推理速度上提高了30倍以上，并在内存消耗上减少了75%。
英文摘要：
We introduce ARPG, a novel visual Autoregressive model that enables Randomized Parallel Generation, addressing the inherent limitations of conventional raster-order approaches, which hinder inference efficiency and zero-shot generalization due to their sequential, predefined token generation order. Our key insight is that effective random-order modeling necessitates explicit guidance for determining the position of the next predicted token. To this end, we propose a novel decoupled decoding framework that decouples positional guidance from content representation, encoding them separately as queries and key-value pairs. By directly incorporating this guidance into the causal attention mechanism, our approach enables fully random-order training and generation, eliminating the need for bidirectional attention. Consequently, ARPG readily generalizes to zero-shot tasks such as image in-painting, out-painting, and resolution expansion. Furthermore, it supports parallel inference by concurrently processing multiple queries using a shared KV cache. On the ImageNet-1K 256 benchmark, our approach attains an FID of 1.83 with only 32 sampling steps, achieving over a 30 times speedup in inference and and a 75 percent reduction in memory consumption compared to representative recent autoregressive models at a similar scale.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
