Seg-VAR：基于视觉自回归建模的图像分割
Seg-VAR: Image Segmentation with Visual Autoregressive Modeling
AI总结：
针对视觉自回归建模在分割任务的潜力未被挖掘的问题，提出Seg-VAR框架将分割转化为条件自回归掩码生成任务，经多阶段训练后在多类分割基准上超越现有方法。
摘要：
尽管视觉自回归建模（VAR）策略已借助自回归模型为图像生成领域带来突破，但其在需要精准低层空间感知的分割任务上的潜力尚未得到探索。受经典基于Mask2Former的模型的多尺度建模思路启发，本文提出Seg-VAR这一全新框架，将分割任务重新定义为条件自回归掩码生成问题，通过将判别式学习替换为隐变量学习流程实现该目标。具体而言，该方法包含三大核心组件：（1）从输入图像生成隐变量先验的图像编码器；（2）空间感知seglat（分割掩码的隐变量表达）编码器，利用位置敏感的颜色映射区分不同实例，将分割掩码映射为离散隐变量token；（3）从上述隐变量重建掩码的解码器。本文还提出多阶段训练策略：首先通过图像-seglat联合训练学习seglat表征，随后优化隐变量变换过程，最终将图像编码器生成的隐变量与seglat分布对齐。实验结果表明，Seg-VAR在各类分割任务和验证基准上的表现均优于此前的判别式方法与生成式方法。通过将分割构建为序列层级预测任务，Seg-VAR为将自回归推理融入空间感知视觉系统开辟了新路径，代码将开源至https://github.com/rkzheng99/Seg-VAR。
英文摘要：
While visual autoregressive modeling (VAR) strategies have shed light on image generation with the autoregressive models, their potential for segmentation, a task that requires precise low-level spatial perception, remains unexplored. Inspired by the multi-scale modeling of classic Mask2Former-based models, we propose Seg-VAR, a novel framework that rethinks segmentation as a conditional autoregressive mask generation problem. This is achieved by replacing the discriminative learning with the latent learning process. Specifically, our method incorporates three core components: (1) an image encoder generating latent priors from input images, (2) a spatial-aware seglat (a latent expression of segmentation mask) encoder that maps segmentation masks into discrete latent tokens using a location-sensitive color mapping to distinguish instances, and (3) a decoder reconstructing masks from these latents. A multi-stage training strategy is introduced: first learning seglat representations via image-seglat joint training, then refining latent transformations, and finally aligning image-encoder-derived latents with seglat distributions. Experiments show Seg-VAR outperforms previous discriminative and generative methods on various segmentation tasks and validation benchmarks. By framing segmentation as a sequential hierarchical prediction task, Seg-VAR opens new avenues for integrating autoregressive reasoning into spatial-aware vision systems. Code will be available at https://github.com/rkzheng99/Seg-VAR.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
