发表机构
RUC; HKUST(GZ); NUS; UCD（中国人民大学; 香港科技大学（广州）; 新加坡国立大学; 加州大学戴维斯分校）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出ToolArtist，一种全智能体图像生成模型，通过后训练UMM实现，采用RAD-GRPO方法优化，将完整开放世界图像生成过程置于智能体控制下，性能优于部分控制方案。
AI 中文摘要
文本到图像（T2I）模型可生成视觉效果出色的图像，但在需要复杂语义理解、多步推理及整合外部世界知识的开放世界任务中仍存在局限。现有研究将智能体能力引入图像生成领域，却要么预设固定工作流程，要么仅将开放世界图像生成过程的一部分置于智能体控制之下，导致推理、工具调用与图像生成无法由单一策略协调。本文提出ToolArtist，这是一种通过对统一多模态模型（UMM）进行后训练得到的全智能体图像生成模型，它可在单一统一策略内动态协调推理、外部工具使用及原生图像生成。在监督微调（SFT）阶段，我们为教师智能体配备搜索工具与图像生成工具，随后将收集到的轨迹转换为UMM兼容格式，其中隐藏图像生成工具但保留生成的图像。在强化学习（RL）阶段，我们开发了适用于UMM的智能体RL基础设施，并提出推理-行动-绘制GRPO（RAD-GRPO），该方法利用互补的意图奖励与质量奖励联合优化模型。实验表明，将整个开放世界图像生成过程置于智能体策略控制下，其性能始终优于采用固定流程或仅部分组件由智能体控制的方法。我们发布了训练数据及完整的后训练基础设施。
英文摘要
Text-to-image (T2I) models can produce visually compelling images, yet they remain limited on open-world tasks that require complex semantic understanding, multi-step reasoning, and the integration of external world knowledge. Existing efforts introduce agent capabilities into image generation, but they either prescribe a fixed workflow or place only a subset of the open-world image generation process under agent control. Consequently, reasoning, tool invocation, and image generation are not coordinated by a single policy. We propose ToolArtist, a fully agentic image generation model obtained by post-training a Unified Multimodal Model (UMM). ToolArtist dynamically orchestrates reasoning, external tool use, and native image generation within one unified policy. During Supervised Fine-Tuning (SFT), we equip a teacher agent with search tools alongside an image-generation tool. We then convert the collected trajectories into a UMM compatible format, where the image-generation tool is concealed while the resulting generated images are retained. During Reinforcement Learning (RL), we develop an agentic RL infrastructure for UMMs and introduce Reason-Act-Draw GRPO (RAD-GRPO), which uses complementary intent and quality rewards to jointly optimize the model. Experiments show that placing the entire open-world image-generation process under an agent policy consistently outperforms approaches with fixed pipelines or only partially agent-controlled components. We release the training data and the complete post-training infrastructure.
