UniCanvas: 一种基于扩散的图文联合生成统一模型
UniCanvas: A Diffusion-base Unified Model for Text-in-Image Joint Generation
AI总结：
提出UniCanvas，通过扩散模型在像素画布上以文本嵌入图像的方式实现图文联合生成，解决现有模型在视觉与文本生成上的不足。
摘要：
近年来，在单一架构内同时处理多模态理解与生成的统一视觉语言模型取得了显著进展。虽然自回归VLM能够跨模态推理，但无法生成高质量图像。相比之下，扩散模型能生成逼真的视觉效果，却难以生成连贯的文本，这使得开发一个能无缝处理视觉和文本生成的统一模型变得具有挑战性。最近的进展表明，语言可以有效地嵌入到视觉表示中，使模型能够直接从图像中推理文本语义。为此，我们提出了UniCanvas，这是首次尝试通过文本图像生成来统一扩散模型以生成交错多模态内容。扩散模型自然地捕捉共享像素画布上的变换，这可以视为视觉变化的世界模型。该模型不是生成离散的文本标记，而是学习将语言表示为图像内部的视觉模式，利用其固有的多模态嵌入空间。这种设计使得模型在图像合成过程中能够在单个像素画布上自然地“绘制”文本，实现无缝的多模态生成。实验表明，UniCanvas在性能上优于先前的统一模型，将基于扩散模型的文本图像生成定位为一种有前景的统一多模态生成范式。
英文摘要：
Recent years have seen remarkable progress in unified vision-language models handling both multimodal understanding and generation within a single architecture. While autoregressive VLMs can reason across modalities, they fail to generate high-quality images. In contrast, diffusion models produce photorealistic visuals yet struggle to generate coherent text, making it challenging to develop a single unified model that can seamlessly handle both visual and text generation. Recent advances suggest that language can be effectively embedded within visual representations, allowing models to reason about textual semantics directly from images. To this end, we propose UniCanvas, a first attempt that unifies diffusion models to generate interleaved multimodal contents through text-in-image generation. Diffusion models naturally capture transformations on a shared pixel canvas, which can be viewed as world models of visual change. Instead of producing discrete text tokens, the model learns to represent language as visual patterns inside images, leveraging its inherent multimodal embedding space. This design allows the model to "draw" text naturally within a single pixel canvas during image synthesis, achieving seamless multimodal generation. Experiments demonstrate that UniCanvas improves performance over previous unified models, positioning text-in-image generation with diffusion models as a promising unified multimodal generation paradigm.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
