GPTFace：通过跨度掩码和弱相关文本-图像数据进行面部-语言 Transformer 的生成式预训练
GPTFace: Generative Pre-training of Facial-Linguistic Transformer by Span Masking and Weakly Correlated Text-image Data
AI总结：
针对面部知识预训练依赖人工标注数据的问题，本文提出 GPTFace，利用弱相关网络图文数据进行 MILM 与 ITM 自监督生成式预训练，并在下游识别和面部编辑任务上达到先进性能。
摘要：
与自然图像理解中预训练模型的繁荣相比，面向面部知识学习的大规模预训练模型研究仍然有限。当前方法主要依赖人工组装和标注的面部数据集进行训练，但标注此类数据集耗费人力，且训练后的模型在训练数据之外的可扩展性有限。为解决这些局限，我们提出了一种用于面部知识学习的生成式预训练模型，利用大规模网络构建数据进行训练。我们使用从互联网爬取的包含人脸的文本和图像，并在包括掩码图像/语言建模（MILM）和图像-文本匹配（ITM）在内的自监督任务上开展预训练。在生成阶段，我们进一步利用图像-文本匹配损失，将生成分布拉向控制信号，以实现可控的图像/文本生成。实验结果表明，在属性分类和表情识别等多种面部下游任务中，我们的模型取得了与最先进预训练模型相当的性能。此外，我们的方法还适用于广泛的面部编辑任务，包括面部属性编辑、表情操控、口罩移除和照片修复。
英文摘要：
Compared to the prosperity of pre-training models in natural image understanding, the research on large-scale pre-training models for facial knowledge learning is still limited. Current approaches mainly rely on manually assembled and annotated face datasets for training, but labeling such datasets is labor-intensive and the trained models have limited scalability beyond the training data. To address these limitations, we present a generative pre-training model for facial knowledge learning that leverages large-scale web-built data for training. We use texts and images containing human faces crawled from the internet and conduct pre-training on self-supervised tasks, including masked image/language modeling (MILM) and image-text matching (ITM). During the generation stage, we further utilize the image-text matching loss to pull the generation distribution towards the control signal for controllable image/text generation. Experimental results demonstrate that our model achieves comparable performance to state-of-the-art pre-training models for various facial downstream tasks, such as attribution classification and expression recognition. Furthermore, our approach is also applicable to a wide range of face editing tasks, including face attribute editing, expression manipulation, mask removal, and photo inpainting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
