UniModel: 一种仅依赖视觉的统一多模态理解和生成框架
UniModel: A Visual-Only Framework for Unified Multimodal Understanding and Generation
AI总结：
UniModel通过统一模型、任务和表示，在单一视觉空间中实现多模态理解和生成的统一框架。
摘要：
我们提出了UniModel，一种统一的生成模型，它在单一的像素到像素扩散框架内同时支持视觉理解和视觉生成。我们的目标是沿三个轴实现统一：模型、任务和表示。在表示层面，我们通过将文本和图像映射到共享的视觉空间来消除模态差异：文本提示被渲染为在干净画布上的绘画文本图像，所有输入和输出都被纯粹视为RGB像素。这产生了完全视觉原生的多模态学习形式。在任务层面，广泛范围的视觉语言问题都被转化为该视觉空间中的像素到像素转换。对于理解任务，模型接收一个RGB图像并生成一个视觉编码语义预测的绘画文本图像。对于生成任务，绘画文本图像作为视觉条件，指导生成逼真且语义对齐的图像合成。描述和文本到图像生成因此成为同一基础视觉翻译过程的不同方向。在模型层面，我们实例化了一个单统一扩散变压器，通过像素空间中的校正流进行训练。一个共享的主干共同学习自然图像和绘画文本图像之间的双向映射，轻量级的任务嵌入用于指定所需的方向。在文本到图像合成和图像到文本理解的实验中，展示了强大的跨模态对齐和涌现的可控性，如循环一致的图像-描述-图像循环。我们的初步探索表明，在单一视觉空间中统一模型、任务和表示是一种有前景的通用多模态智能范式。
英文摘要：
We present UniModel, a unified generative model that jointly supports visual understanding and visual generation within a single pixel-to-pixel diffusion framework. Our goal is to achieve unification along three axes: the model, the tasks, and the representations. At the representation level, we eliminate modality discrepancies by mapping both text and images into a shared visual space: textual prompts are rendered as painted text images on a clean canvas, and all inputs and outputs are treated purely as RGB pixels. This yields a fully vision-native formulation of multimodal learning. At the task level, a broad range of vision-language problems are cast as pixel-to-pixel transformations in this visual space. For understanding tasks, the model takes an RGB image and produces a painted text image that visually encodes the semantic prediction. For generation tasks, painted text images serve as visual conditions that guide realistic and semantically aligned image synthesis. Captioning and text-to-image generation thus become different directions of the same underlying visual translation process. At the model level, we instantiate a single Unified Diffusion Transformer trained with rectified flow in pixel space. A shared backbone jointly learns bidirectional mappings between natural images and painted text images, with lightweight task embeddings to specify the desired direction. Experiments on text-to-image synthesis and image-to-text understanding demonstrate strong cross-modal alignment and emergent controllability such as cycle-consistent image-caption-image loops. Our initial exploration suggests that unifying model, tasks, and representations in a single visual space is a promising paradigm for general-purpose multimodal intelligence.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
