VA-$π$: 变分策略对齐用于像素感知自回归生成
VA-$π$: Variational Policy Alignment for Pixel-Aware Autoregressive Generation
AI总结：
VA-$π$通过变分优化和像素空间目标提升自回归生成的质量和多样性。
摘要：
自回归（AR）视觉生成依赖于分词器将图像映射到和从离散序列。然而，分词器是训练以从真实令牌重建干净图像，而AR生成器仅优化令牌似然。这种不一致导致生成的令牌序列可能解码成低质量图像，没有来自像素空间的直接监督。我们提出了VA-$π$，一个轻量级的后训练框架，直接优化AR模型，以原则性的像素空间目标。VA-$π$将生成器-分词器对齐公式化为变分优化，推导出一个证据下界（ELBO），统一了像素重建和自回归建模。为了在离散令牌空间下优化，VA-$π$引入了一种基于强化学习的对齐策略，将AR生成器视为策略，使用像素空间重建质量作为其内在奖励。奖励通过预测令牌序列在教师强制下的重建能力来测量，给模型提供直接的像素级指导，而无需昂贵的自由运行采样。ELBO的正则化项作为自然正则化器，保持令牌的分布一致性。VA-$π$使现有AR生成器能够快速适应，无需重新训练分词器或外部奖励模型。仅使用1%的ImageNet-1K数据和25分钟的调优，它将FID从14.36降低到7.65，并将IS从86.55提高到116.70，在LlamaGen-XXL上，同时在GenEval上的文本到图像任务中也产生了显著的提升，对于视觉生成模型（LlamaGen：从0.306到0.339）和统一多模态模型（Janus-Pro：从0.725到0.744）。代码可在https://github.com/Lil-Shake/VA-Pi上获得。
英文摘要：
Autoregressive (AR) visual generation relies on tokenizers to map images to and from discrete sequences. However, tokenizers are trained to reconstruct clean images from ground-truth tokens, while AR generators are optimized only for token likelihood. This misalignment leads to generated token sequences that may decode into low-quality images, without direct supervision from the pixel space. We propose VA-$π$, a lightweight post-training framework that directly optimizes AR models with a principled pixel-space objective. VA-$π$ formulates the generator-tokenizer alignment as a variational optimization, deriving an evidence lower bound (ELBO) that unifies pixel reconstruction and autoregressive modeling. To optimize under the discrete token space, VA-$π$ introduces a reinforcement-based alignment strategy that treats the AR generator as a policy, uses pixel-space reconstruction quality as its intrinsic reward. The reward is measured by how well the predicted token sequences can reconstruct the original image under teacher forcing, giving the model direct pixel-level guidance without expensive free-running sampling. The regularization term of the ELBO serves as a natural regularizer, maintaining distributional consistency of tokens. VA-$π$ enables rapid adaptation of existing AR generators, without neither tokenizer retraining nor external reward models. With only 1% ImageNet-1K data and 25 minutes of tuning, it reduces FID from 14.36 to 7.65 and improves IS from 86.55 to 116.70 on LlamaGen-XXL, while also yielding notable gains in the text-to-image task on GenEval for both visual generation model (LlamaGen: from 0.306 to 0.339) and unified multi-modal model (Janus-Pro: from 0.725 to 0.744). Code is available at https://github.com/Lil-Shake/VA-Pi.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
