arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.07293cs.CV
收藏

推理时的缩放用于视觉自回归建模通过搜索代表性样本

Inference-Time Scaling for Visual AutoRegressive modeling by Searching Representative Samples

School of Electronic Engineering, Xidian University, Xi'an, China(西安电子科技大学电子工程学院)

Weidong Tang, Xinyan Wan, Siyu Li, Xiumei Wang

AI总结:

本文提出VAR-Scaling框架,通过搜索代表性样本解决VQ视觉自回归建模中的离散潜在空间问题,提升推理过程的生成质量。

摘要:

尽管推理时的缩放显著提高了大型语言和扩散模型的生成质量,但将其应用于向量量化(VQ)视觉自回归建模(VAR)仍处于未探索的领域。我们引入了VAR-缩放,这是第一个用于VAR中的推理时缩放的一般框架,解决了离散潜在空间中禁止连续路径搜索的关键挑战。我们发现VAR缩放表现出两种不同的模式类型:通用模式和特定模式,其中后期阶段的特定模式有条件地优化早期阶段的通用模式。为了克服VQ模型中的离散潜在空间障碍,我们通过核密度估计(KDE)将采样空间映射到近似连续的特征空间,其中高密度样本近似稳定、高质量的解决方案。这种转换使能够有效地导航采样分布。我们提出了一种密度自适应的混合采样策略:Top-k采样专注于高密度区域以在分布模式附近保持质量,而Random-k采样探索低密度区域以保持多样性并防止过早收敛。因此,VAR-缩放优化关键尺度的样本保真度以提高输出质量。在类别条件和文本到图像评估中的实验表明,在推理过程中有显著的改进。代码可在https://github.com/WD7ang/VAR-Scaling上获得。

英文摘要:

While inference-time scaling has significantly enhanced generative quality in large language and diffusion models, its application to vector-quantized (VQ) visual autoregressive modeling (VAR) remains unexplored. We introduce VAR-Scaling, the first general framework for inference-time scaling in VAR, addressing the critical challenge of discrete latent spaces that prohibit continuous path search. We find that VAR scales exhibit two distinct pattern types: general patterns and specific patterns, where later-stage specific patterns conditionally optimize early-stage general patterns. To overcome the discrete latent space barrier in VQ models, we map sampling spaces to quasi-continuous feature spaces via kernel density estimation (KDE), where high-density samples approximate stable, high-quality solutions. This transformation enables effective navigation of sampling distributions. We propose a density-adaptive hybrid sampling strategy: Top-k sampling focuses on high-density regions to preserve quality near distribution modes, while Random-k sampling explores low-density areas to maintain diversity and prevent premature convergence. Consequently, VAR-Scaling optimizes sample fidelity at critical scales to enhance output quality. Experiments in class-conditional and text-to-image evaluations demonstrate significant improvements in inference process. The code is available at https://github.com/WD7ang/VAR-Scaling.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑