ControlVAR：探索可控视觉自回归建模
ControlVAR: Exploring Controllable Visual Autoregressive Modeling
AI总结：
本文提出 ControlVAR 框架，通过联合建模图像与像素级条件、下一尺度 AR 预测及教师强制引导，实现比主流条件扩散模型更灵活高效的可控视觉生成。
摘要：
随着扩散模型（DMs）的出现，条件视觉生成取得了显著进展，尤其是在控制到图像生成等任务中。然而，计算成本高昂、推理延迟较高以及难以与大语言模型（LLMs）集成等挑战，使得探索 DMs 的替代方案成为必要。本文提出 ControlVAR，这是一个新颖框架，在视觉自回归（VAR）建模中探索像素级控制，以实现灵活且高效的条件生成。与学习条件分布的传统条件模型不同，ControlVAR 在训练期间联合建模图像与像素级条件的分布，并在测试期间施加条件控制。为增强联合建模，我们采用下一尺度 AR 预测范式，并统一控制表示与图像表示。本文还提出教师强制引导策略，以进一步促进联合建模下的可控生成。大量实验表明，相较于 ControlNet 和 T2I-Adaptor 等流行条件 DMs，ControlVAR 在多种条件生成任务上具有更优的有效性和灵活性。代码：https://github.com/lxa9867/ControlVAR。
英文摘要：
Conditional visual generation has witnessed remarkable progress with the advent of diffusion models (DMs), especially in tasks like control-to-image generation. However, challenges such as expensive computational cost, high inference latency, and difficulties of integration with large language models (LLMs) have necessitated exploring alternatives to DMs. This paper introduces ControlVAR, a novel framework that explores pixel-level controls in visual autoregressive (VAR) modeling for flexible and efficient conditional generation. In contrast to traditional conditional models that learn the conditional distribution, ControlVAR jointly models the distribution of image and pixel-level conditions during training and imposes conditional controls during testing. To enhance the joint modeling, we adopt the next-scale AR prediction paradigm and unify control and image representations. A teacher-forcing guidance strategy is proposed to further facilitate controllable generation with joint modeling. Extensive experiments demonstrate the superior efficacy and flexibility of ControlVAR across various conditional generation tasks against popular conditional DMs, \eg, ControlNet and T2I-Adaptor. Code: \url{https://github.com/lxa9867/ControlVAR}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
