FontCLIP:面向多语言字体应用的语义排版视觉-语言模型
FontCLIP: A Semantic Typography Visual-Language Model for Multilingual Font Applications
AI总结:
本文提出 FontCLIP,通过复合属性提示微调 CLIP 以融合排版知识,实现跨 CJK 语言及训练外语义属性的字体检索与字形优化。
摘要:
为各类设计任务获取所需字体颇具挑战,并且需要专业的排版知识。尽管以往的字体检索或生成工作已经缓解了其中部分困难,但它们通常缺乏对多种语言以及训练数据领域之外语义属性的支持。为解决这一问题,我们提出了 FontCLIP:一个将大型视觉-语言模型的语义理解与排版知识相连接的模型。我们通过一种新颖的微调方法,将特定于排版的知识整合到预训练 CLIP 模型全面的视觉-语言知识中。我们提议使用一种复合描述性提示,该提示封装了从一个聚焦于罗马字母字符的字体属性数据集中自适应采样得到的属性。FontCLIP 的语义排版潜在空间展现出两种前所未有的泛化能力。首先,FontCLIP 可泛化到包括中文、日文和韩文(CJK)在内的不同语言,捕捉跨不同语言字体的排版特征,即便它仅使用罗马字符字体进行了微调。其次,FontCLIP 能够识别训练数据中未出现的语义属性。FontCLIP 的双模态与泛化能力支持多语言和跨语言字体检索以及字形优化,减轻了获取所需字体的负担。
英文摘要:
Acquiring the desired font for various design tasks can be challenging and requires professional typographic knowledge. While previous font retrieval or generation works have alleviated some of these difficulties, they often lack support for multiple languages and semantic attributes beyond the training data domains. To solve this problem, we present FontCLIP: a model that connects the semantic understanding of a large vision-language model with typographical knowledge. We integrate typography-specific knowledge into the comprehensive vision-language knowledge of a pretrained CLIP model through a novel finetuning approach. We propose to use a compound descriptive prompt that encapsulates adaptively sampled attributes from a font attribute dataset focusing on Roman alphabet characters. FontCLIP's semantic typographic latent space demonstrates two unprecedented generalization abilities. First, FontCLIP generalizes to different languages including Chinese, Japanese, and Korean (CJK), capturing the typographical features of fonts across different languages, even though it was only finetuned using fonts of Roman characters. Second, FontCLIP can recognize the semantic attributes that are not presented in the training data. FontCLIP's dual-modality and generalization abilities enable multilingual and cross-lingual font retrieval and letter shape optimization, reducing the burden of obtaining desired fonts.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。