FontStudio：面向连贯一致字体效果生成的形状自适应扩散模型
FontStudio: Shape-Adaptive Diffusion Model for Coherent and Consistent Font Effect Generation
AI总结：
本文提出FontStudio形状自适应扩散模型，通过分割掩码引导不规则字体画布内的图像生成，并利用无需训练的效果迁移方法保持多字母一致性，在用户研究中以78%美学胜率超越Adobe Firefly。
摘要：
近年来，将现代基于扩散的文本到图像生成模型应用于艺术字体创作——这一传统上属于专业设计师的领域——已引起广泛关注。与大多数现有研究集中于生成艺术字体排版不同，本研究旨在解决一个新颖且更具挑战性的问题：多语言字体的文本效果生成。该任务本质上要求在字体形状的画布范围内生成连贯且一致的视觉内容，而非传统的矩形画布。为解决这一问题，我们提出了一种新颖的形状自适应扩散模型，该模型能够理解给定的形状，并在不规则画布内策略性地规划像素分布。为此，我们构建了一个高质量的形状自适应图像-文本数据集，并将分割掩码作为视觉条件引入，以引导不规则画布内的图像生成过程。该方法使传统基于矩形画布的扩散模型能够根据给定的几何形状生成所需的概念。其次，为保持多个字母之间的一致性，我们还提出了一种无需训练的形状自适应效果迁移方法，用于将纹理从生成的参考字母迁移到其他字母。其关键思路在于构建字体效果噪声先验，并在拼接的潜在空间中传播字体效果信息。FontStudio系统的有效性通过用户偏好研究得到证实，研究显示，即使与最新的无与伦比的商业产品Adobe Firefly相比，我们的系统也获得了显著偏好（美学方面78%的胜率）。
英文摘要：
Recently, the application of modern diffusion-based text-to-image generation models for creating artistic fonts, traditionally the domain of professional designers, has garnered significant interest. Diverging from the majority of existing studies that concentrate on generating artistic typography, our research aims to tackle a novel and more demanding challenge: the generation of text effects for multilingual fonts. This task essentially requires generating coherent and consistent visual content within the confines of a font-shaped canvas, as opposed to a traditional rectangular canvas. To address this task, we introduce a novel shape-adaptive diffusion model capable of interpreting the given shape and strategically planning pixel distributions within the irregular canvas. To achieve this, we curate a high-quality shape-adaptive image-text dataset and incorporate the segmentation mask as a visual condition to steer the image generation process within the irregular-canvas. This approach enables the traditionally rectangle canvas-based diffusion model to produce the desired concepts in accordance with the provided geometric shapes. Second, to maintain consistency across multiple letters, we also present a training-free, shape-adaptive effect transfer method for transferring textures from a generated reference letter to others. The key insights are building a font effect noise prior and propagating the font effect information in a concatenated latent space. The efficacy of our FontStudio system is confirmed through user preference studies, which show a marked preference (78% win-rates on aesthetics) for our system even when compared to the latest unrivaled commercial product, Adobe Firefly.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
