Skywork UniPic 2.0: 通过在线强化学习构建 Kontext 模型以实现统一多模态模型
Skywork UniPic 2.0: Building Kontext Model with Online RL for Unified Multimodal Model
AI总结：
Skywork UniPic 2.0 通过在线强化学习构建 Kontext 模型，实现统一多模态模型，提升图像生成与编辑能力。
摘要：
近年来，多模态模型的进展展示了在统一图像生成和编辑方面的 impressive 能力。然而，许多著名的开源模型更注重扩大模型参数而非优化训练策略，限制了其效率和性能。在本工作中，我们提出了 UniPic2-SD3.5M-Kontext，这是一个基于 SD3.5-Medium 的 2B 参数 DiT 模型，实现了 state-of-the-art 的图像生成和编辑，同时无缝扩展到统一的多模态框架。我们的方法首先对 SD3.5-Medium 进行架构修改，并在高质量数据上进行大规模预训练，以实现文本到图像生成和编辑能力。为了增强指令遵循和编辑一致性，我们提出了一种新的渐进双任务强化策略（PDTR），该策略以阶段方式有效加强了两个任务。我们实证验证了不同任务的强化阶段是相互有益的，且不会引起负面影响。在预训练和强化策略之后，UniPic2-SD3.5M-Kontext 的图像生成和编辑能力比具有显著更大生成参数的模型（包括 BAGEL（7B）和 Flux-Kontext（12B））更强。此外，遵循 MetaQuery，我们通过连接器将 UniPic2-SD3.5M-Kontext 和 Qwen2.5-VL-7B 连接起来，并进行联合训练，以启动统一的多模态模型 UniPic2-Metaquery。UniPic2-Metaquery 集成了理解和生成与编辑，通过简单且可扩展的训练范式，在多样任务中实现了顶尖性能。这始终验证了我们所提出训练范式的有效性与通用性，我们将其正式称为 Skywork UniPic 2.0。
英文摘要：
Recent advances in multimodal models have demonstrated impressive capabilities in unified image generation and editing. However, many prominent open-source models prioritize scaling model parameters over optimizing training strategies, limiting their efficiency and performance. In this work, we present UniPic2-SD3.5M-Kontext, a 2B-parameter DiT model based on SD3.5-Medium, which achieves state-of-the-art image generation and editing while extending seamlessly into a unified multimodal framework. Our approach begins with architectural modifications to SD3.5-Medium and large-scale pre-training on high-quality data, enabling joint text-to-image generation and editing capabilities. To enhance instruction following and editing consistency, we propose a novel Progressive Dual-Task Reinforcement strategy (PDTR), which effectively strengthens both tasks in a staged manner. We empirically validate that the reinforcement phases for different tasks are mutually beneficial and do not induce negative interference. After pre-training and reinforcement strategies, UniPic2-SD3.5M-Kontext demonstrates stronger image generation and editing capabilities than models with significantly larger generation parameters-including BAGEL (7B) and Flux-Kontext (12B). Furthermore, following the MetaQuery, we connect the UniPic2-SD3.5M-Kontext and Qwen2.5-VL-7B via a connector and perform joint training to launch a unified multimodal model UniPic2-Metaquery. UniPic2-Metaquery integrates understanding, generation, and editing, achieving top-tier performance across diverse tasks with a simple and scalable training paradigm. This consistently validates the effectiveness and generalizability of our proposed training paradigm, which we formalize as Skywork UniPic 2.0.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
