UTDesign: 一种统一框架，用于图形设计图像中的风格化文本编辑与生成
UTDesign: A Unified Framework for Stylized Text Editing and Generation in Graphic Design Images
AI总结：
UTDesign提出了一种统一框架，实现高精度风格化文本编辑与生成，支持中英文文本，通过多模态编码器和DiT模型提升设计图像中的文本生成与编辑能力。
摘要：
人工智能辅助的图形设计已成为自动化创建和编辑设计元素（如海报、横幅和广告）的强大工具。尽管基于扩散的文本到图像模型在视觉内容生成中表现出强大的能力，但其在小规模字体和非拉丁文字的文本渲染性能仍然有限。在本文中，我们提出UTDesign，一种用于设计图像中高精度风格化文本编辑和条件文本生成的统一框架，支持英语和中文文字。我们的框架引入了一个基于DiT的新型文本风格迁移模型，该模型从头开始在合成数据集上进行训练，能够生成保持参考字形风格的透明RGBA文本前景。我们进一步将该模型扩展为一个条件文本生成框架，通过在经过精心挑选的数据集上训练一个多模态条件编码器，该数据集具有详细的文本注释，使能够根据背景图像、提示和布局规范生成准确且风格一致的文本。最后，我们将我们的方法整合到一个完全自动化的文本到设计（T2D）管道中，通过结合预训练的文本到图像（T2I）模型和基于MLLM的布局规划器。广泛的实验表明，UTDesign在开放源方法中在风格一致性和文本准确性方面实现了最先进的性能，并且在与专有商业方法相比时也表现出独特的优势。本文的代码和数据可在https://github.com/ZYM-PKU/UTDesign上获得。
英文摘要：
AI-assisted graphic design has emerged as a powerful tool for automating the creation and editing of design elements such as posters, banners, and advertisements. While diffusion-based text-to-image models have demonstrated strong capabilities in visual content generation, their text rendering performance, particularly for small-scale typography and non-Latin scripts, remains limited. In this paper, we propose UTDesign, a unified framework for high-precision stylized text editing and conditional text generation in design images, supporting both English and Chinese scripts. Our framework introduces a novel DiT-based text style transfer model trained from scratch on a synthetic dataset, capable of generating transparent RGBA text foregrounds that preserve the style of reference glyphs. We further extend this model into a conditional text generation framework by training a multi-modal condition encoder on a curated dataset with detailed text annotations, enabling accurate, style-consistent text synthesis conditioned on background images, prompts, and layout specifications. Finally, we integrate our approach into a fully automated text-to-design (T2D) pipeline by incorporating pre-trained text-to-image (T2I) models and an MLLM-based layout planner. Extensive experiments demonstrate that UTDesign achieves state-of-the-art performance among open-source methods in terms of stylistic consistency and text accuracy, and also exhibits unique advantages compared to proprietary commercial approaches. Code and data for this paper are available at https://github.com/ZYM-PKU/UTDesign.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
