EditAR：基于自回归模型的统一条件生成
EditAR: Unified Conditional Generation with Autoregressive Models
AI总结：
本文提出统一自回归框架 EditAR，将图像与指令作为条件，通过下一 token 预测完成图像编辑及深度、边缘、分割等条件生成，并蒸馏基础模型知识以增强图文对齐。
摘要：
近年来，可控图像生成与编辑的进展在很大程度上由基于扩散的方法推动。尽管扩散模型在采用定制设计的特定任务中表现极其出色，但构建统一模型仍具挑战。相比之下，自回归模型天然具有统一的分词表示特征，这简化了为多种任务创建单一基础模型的过程。在本研究中，我们提出 EditAR，这是一个单一统一的自回归框架，适用于多种条件图像生成任务，例如图像编辑、深度图转图像、边缘转图像、分割转图像。该模型同时接收图像和指令作为输入，并以普通的下一 token 预测范式预测编辑后图像的 token。为增强文本到图像的对齐，我们进一步提出将基础模型的知识蒸馏到自回归建模过程中。我们在既有基准上跨多种任务评估了其有效性，结果显示其性能与各类最先进的任务专用方法相比具有竞争力。项目主页：https://jitengmu.github.io/EditAR/
英文摘要：
Recent progress in controllable image generation and editing is largely driven by diffusion-based methods. Although diffusion models perform exceptionally well in specific tasks with tailored designs, establishing a unified model is still challenging. In contrast, autoregressive models inherently feature a unified tokenized representation, which simplifies the creation of a single foundational model for various tasks. In this work, we propose EditAR, a single unified autoregressive framework for a variety of conditional image generation tasks, e.g., image editing, depth-to-image, edge-to-image, segmentation-to-image. The model takes both images and instructions as inputs, and predicts the edited images tokens in a vanilla next-token paradigm. To enhance the text-to-image alignment, we further propose to distill the knowledge from foundation models into the autoregressive modeling process. We evaluate its effectiveness across diverse tasks on established benchmarks, showing competitive performance to various state-of-the-art task-specific methods. Project page: https://jitengmu.github.io/EditAR/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
