arXiv 2502.203132026-01-12cs.CV

FlexVAR: 无需残差预测的灵活视觉自回归建模

FlexVAR: Flexible Visual Autoregressive Modeling without Residual Prediction

Institute of Information Science, Beijing Jiaotong University（北京交通大学信息科学学院）；University of Technology Sydney（悉尼科技大学）；Meituan（美团）；Georgia Institute of Technology（佐治亚理工学院）

Siyu Jiao, Gengwei Zhang, Yinlong Qian, Jiancheng Huang, Yao Zhao, Humphrey Shi, Lin Ma, Yunchao Wei, Zequn Jie

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

FlexVAR通过无需残差预测的灵活视觉自回归建模，在低分辨率图像上实现高分辨率图像生成和多种图像任务处理。

摘要：

本文挑战了视觉自回归建模中的残差预测范式，提出了FlexVAR，一种新的灵活视觉自回归图像生成范式。FlexVAR通过真实预测实现自回归学习，使每一步都能独立生成合理的图像。这种简单直观的方法能够迅速学习视觉分布，使生成过程更加灵活和适应性强。仅在低分辨率图像（≤256px）上训练，FlexVAR可以：（1）生成各种分辨率和纵横比的图像，甚至超过训练图像的分辨率。（2）支持多种图像到图像任务，包括图像细化、内外修补和图像扩展。（3）适应各种自回归步骤，允许通过更少的步骤实现更快的推理，或通过更多的步骤提高图像质量。我们的1.0B模型在ImageNet 256×256基准上优于其VAR对应物。此外，当使用13步进行零样本迁移时，性能进一步提升到2.08 FID，优于最先进的自回归模型AiM/VAR（0.25/0.28 FID）和流行的扩散模型LDM/DiT（1.52/0.19 FID）。当将我们的1.0B模型迁移至ImageNet 512×512基准时，FlexVAR在与完全监督的VAR 2.3B模型竞争中取得了具有竞争力的结果。

英文摘要：

This work challenges the residual prediction paradigm in visual autoregressive modeling and presents FlexVAR, a new Flexible Visual AutoRegressive image generation paradigm. FlexVAR facilitates autoregressive learning with ground-truth prediction, enabling each step to independently produce plausible images. This simple, intuitive approach swiftly learns visual distributions and makes the generation process more flexible and adaptable. Trained solely on low-resolution images ($\leq$ 256px), FlexVAR can: (1) Generate images of various resolutions and aspect ratios, even exceeding the resolution of the training images. (2) Support various image-to-image tasks, including image refinement, in/out-painting, and image expansion. (3) Adapt to various autoregressive steps, allowing for faster inference with fewer steps or enhancing image quality with more steps. Our 1.0B model outperforms its VAR counterpart on the ImageNet 256$\times$256 benchmark. Moreover, when zero-shot transfer the image generation process with 13 steps, the performance further improves to 2.08 FID, outperforming state-of-the-art autoregressive models AiM/VAR by 0.25/0.28 FID and popular diffusion models LDM/DiT by 1.52/0.19 FID, respectively. When transferring our 1.0B model to the ImageNet 512$\times$512 benchmark in a zero-shot manner, FlexVAR achieves competitive results compared to the VAR 2.3B model, which is a fully supervised model trained at 512$\times$512 resolution.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。