arXiv 2505.221412025-08-27cs.CVcs.AI

FaceEditTalker：具有面部属性编辑的可控说话头生成

FaceEditTalker: Controllable Talking Head Generation with Facial Attribute Editing

Xi’an Key Laboratory of Big Data and Intelligent Vision, Xidian University, Xi’an 710071, China（西安大数据与智能视觉重点实验室，西安电子科技大学，西安710071，中国）；Key Laboratory of Collaborative Intelligence Systems, Ministry of Education, Xidian University, Xi’an 710071, China（协同智能系统重点实验室，教育部，西安电子科技大学，西安710071，中国）；School of Computer Science and Technology, Xidian University, Xi’an 710071, China（计算机科学与技术学院，西安电子科技大学，西安710071，中国）

Guanwen Feng, Zhiyuan Ma, Yunan Li, Jiahao Yang, Junwei Jing, Qiguang Miao

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出FaceEditTalker统一框架，通过图像特征空间编辑模块和音频驱动视频生成模块，在生成音频同步说话头视频的同时实现表情、发型、配饰等面部属性的可控编辑，并在唇形同步、视频质量和属性可控性上达到与代表性基线方法相当或更优的性能。

摘要：

近年来，音频驱动的说话头生成在唇形同步和情感表达方面取得了令人瞩目的成果。然而，现有方法在很大程度上忽视了面部属性编辑这一关键任务。该能力对于实现深度个性化以及拓展实际应用范围不可或缺，包括用户定制数字化身、引人入胜的在线教育内容以及品牌专属数字客服。在这些关键领域中，灵活调整视觉属性（如发型、配饰和细微面部特征）对于契合用户偏好、体现多样化品牌形象以及适应不同情境需求至关重要。本文提出FaceEditTalker，一个统一框架，能够在生成高质量、音频同步的说话头视频的同时实现可控的面部属性操控。我们的方法包含两个关键组件：图像特征空间编辑模块，用于提取语义和细节特征，并允许对表情、发型和配饰等属性进行灵活控制；以及音频驱动视频生成模块，将编辑后的特征与音频引导的面部关键点融合，以驱动基于扩散的生成器。该设计确保了跨帧的时间连贯性、视觉保真度和身份保持。在公开数据集上的大量实验表明，我们的方法在唇形同步精度、视频质量和属性可控性方面达到了与代表性基线方法相当或更优的性能。项目主页：https://peterfanfan.github.io/FaceEditTalker/

英文摘要：

Recent advances in audio-driven talking head generation have achieved impressive results in lip synchronization and emotional expression. However, they largely overlook the crucial task of facial attribute editing. This capability is indispensable for achieving deep personalization and expanding the range of practical applications, including user-tailored digital avatars, engaging online education content, and brand-specific digital customer service. In these key domains, flexible adjustment of visual attributes, such as hairstyle, accessories, and subtle facial features, is essential for aligning with user preferences, reflecting diverse brand identities and adapting to varying contextual demands. In this paper, we present FaceEditTalker, a unified framework that enables controllable facial attribute manipulation while generating high-quality, audio-synchronized talking head videos. Our method consists of two key components: an image feature space editing module, which extracts semantic and detail features and allows flexible control over attributes like expression, hairstyle, and accessories; and an audio-driven video generation module, which fuses these edited features with audio-guided facial landmarks to drive a diffusion-based generator. This design ensures temporal coherence, visual fidelity, and identity preservation across frames. Extensive experiments on public datasets demonstrate that our method achieves comparable or superior performance to representative baseline methods in lip-sync accuracy, video quality, and attribute controllability. Project page: https://peterfanfan.github.io/FaceEditTalker/

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。