There exists recent work in computer vision, named VAR, that proposes a new autoregressive paradigm for image generation. Diverging from the vanilla next-token prediction, VAR structurally reformulates the image generation into a coarse to fine next-scale prediction. In this paper, we show that this scale-wise autoregressive framework can be effectively decoupled into \textit{intra-scale modeling}, which captures local spatial dependencies within each scale, and \textit{inter-scale modeling}, which models cross-scale relationships progressively from coarse-to-fine scales. This decoupling structure allows to rebuild VAR in a more computationally efficient manner. Specifically, for intra-scale modeling -- crucial for generating high-fidelity images -- we retain the original bidirectional self-attention design to ensure comprehensive modeling; for inter-scale modeling, which semantically connects different scales but is computationally intensive, we apply linear-complexity mechanisms like Mamba to substantially reduce computational overhead. We term this new framework M-VAR. Extensive experiments demonstrate that our method outperforms existing models in both image quality and generation speed. For example, our 1.5B model, with fewer parameters and faster inference speed, outperforms the largest VAR-d30-2B. Moreover, our largest model M-VAR-d32 impressively registers 1.78 FID on ImageNet 256$\times$256 and outperforms the prior-art autoregressive models LlamaGen/VAR by 0.4/0.19 and popular diffusion models LDM/DiT by 1.82/0.49, respectively. Code is avaiable at \url{https://github.com/OliverRensu/MVAR}.
CUHK（香港中文大学）；The University of Hong Kong（香港大学）；CUHK (Shenzhen)（香港中文大学（深圳））；Tencent AI Lab（腾讯人工智能实验室）；Adobe Inc.（奥多比公司）
Generative adversarial networks (GANs) have made great success in image inpainting yet still have difficulties tackling large missing regions. In contrast, iterative probabilistic algorithms, such as autoregressive and denoising diffusion models, have to be deployed with massive computing resources for decent effect. To achieve high-quality results with low computational cost, we present a novel pixel spread model (PSM) that iteratively employs decoupled probabilistic modeling, combining the optimization efficiency of GANs with the prediction tractability of probabilistic models. As a result, …
Google Research（谷歌研究院）；CUHK（香港中文大学）；Google Cloud AI（谷歌云人工智能）
We propose Stratified Image Transformer(StraIT), a pure non-autoregressive(NAR) generative model that demonstrates superiority in high-quality image synthesis over existing autoregressive(AR) and diffusion models(DMs). In contrast to the under-exploitation of visual characteristics in existing vision tokenizer, we leverage the hierarchical nature of images to encode visual tokens into stratified levels with emergent properties. Through the proposed image stratification that obtains an interlinked token pair, we alleviate the modeling difficulty and lift the generative power of NAR models. Our …
Epona: Autoregressive Diffusion World Model for Autonomous Driving
Horizon Robotics；Tsinghua University（清华大学）；Peking University（北京大学）；Nanjing University（南京大学）；The Hong Kong University of Science and Technology（香港科学与技术大学）；Nanyang Technological University（南洋理工大学）；Tencent Hunyuan（腾讯文心）
Edit-VAR: Taming Visual Autoregressive Model for Precise Video Editing
Sun Yat-sen University（中山大学）；South China University of Technology（华南理工大学）；Tsinghua University（清华大学）；Shandong University（山东大学）；Nankai University（南开大学）；Hunan University（湖南大学）
MAIS, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所复杂系统管理与控制国家重点实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；VAST（无（暂未找到合适中文名，VAST可音译为“瓦斯特”，但这并非一个正式的中文机构名，所以保留英文））；Tsinghua University（清华大学）；Xi’an Jiaotong University（西安交通大学）
University of Surrey（萨里大学）；iFlyTek-Surrey Joint Research Centre on Artificial Intelligence（科大讯飞-萨里人工智能联合研究中心）
Given an abstract, deformed, ordinary sketch from untrained amateurs like you and me, this paper turns it into a photorealistic image - just like those shown in Fig. 1(a), all non-cherry-picked. We differ significantly from prior art in that we do not dictate an edgemap-like sketch to start with, but aim to work with abstract free-hand human sketches. In doing so, we essentially democratise the sketch-to-photo pipeline, "picturing" a sketch regardless of how good you sketch. Our contribution at the outset is a decoupled encoder-decoder training paradigm, where the decoder is a StyleGAN trained…
Think, then Score: Decoupled Reasoning and Scoring for Video Reward Modeling
University of Science and Technology of China（中国科学技术大学）；Kling Team, Kuaishou Technology（快手科技 Kling 团队）；Institute of Software Chinese Academy of Sciences（中国科学院软件研究所）
NExT-OMNI: Towards Any-to-Any Omnimodal Foundation Models with Discrete Flow Matching
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；NExT++ Research Center（NExT++研究中心）；National University of Singapore（新加坡国立大学）
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；Dalian University of Technology（大连理工大学）；Tsinghua University（清华大学）
针对视觉分词器与下游任务优化脱节导致的表示瓶颈，提出 ETT 端到端联合调优方法，在多模态理解与生成任务上较冻结基线提升 2-6%。
