利用多模态大语言模型增强单图像面部去变形
Enhancing Single-Image Facial Demorphing using Multimodal Large Language Models
AI总结：
提出一种基于多模态大语言模型引导的耦合扩散重建框架，通过提取中间层语义嵌入作为条件，实现无参考的面部去变形，恢复构成图像并保持身份一致性。
摘要：
人脸识别系统越来越容易受到变形攻击，其中合成图像被制作成匹配多个身份，从而实现未经授权的访问和身份欺诈。现有的检测方法可以识别变形图像，但无法恢复构成图像或身份，限制了其取证实用性。本文提出了一种新颖的无参考面部去变形框架，利用多模态大语言模型（MLLMs）引导耦合的扩散重建过程。我们的关键创新在于从MLLM中间层提取语义嵌入以调节去变形过程，提供关于面部属性和身份线索的高级推理，补充低级像素信息。我们将去变形表述为一个耦合的条件生成问题，其中两个构成人脸通过直接在RGB域中操作的去噪扩散模型联合合成，确保身份间一致性，同时保留细粒度的感知细节。与依赖于压缩潜在表示或假设训练集和测试集之间身份重叠的先前方法不同，我们的方法通过直接利用MLLM隐藏状态作为条件信号，绕过了有损的文本生成-重新编码循环，使去噪网络能够关注细微的视觉线索，如头发、背景和面部纹理。消融研究进一步揭示，MLLM中间层编码了更具身份判别性的表示，RGB域去变形在严格操作点上的性能优于潜在空间方法30-40%，并且完整的MLLM嵌入通过多模态预训练的增强语义结构，比原始ViT特征提供了显著优势。
英文摘要：
Face recognition systems are increasingly vulnerable to morphing attacks, where a composite image is crafted to match multiple identities, enabling unauthorized access and identity fraud. Existing detection methods identify morphed images but cannot recover constituent images or identities, limiting their forensic utility. This paper presents a novel reference-free facial demorphing framework that leverages Multimodal Large Language Models (MLLMs) to guide a coupled diffusion-based reconstruction process. Our key innovation lies in extracting semantic embeddings from intermediate MLLM layers to condition the demorphing, providing high-level reasoning about facial attributes and identity cues that complement low-level pixel information. We formulate demorphing as a coupled conditional generation problem, where both constituent faces are synthesized jointly through a denoising diffusion model operating directly in the RGB domain, ensuring inter-identity consistency while preserving fine-grained perceptual details. Unlike prior approaches that rely on compressed latent representations or assume identity overlap between training and testing sets, our method bypasses lossy text generation-reencoding cycles by directly utilizing MLLM hidden states as conditioning signals, enabling the denoising network to attend to subtle visual cues such as hair, background, and facial textures. Ablation studies further reveal that middle MLLM layers encode more identity-discriminative representations, RGB-domain demorphing outperforms latent-space approaches by 30--40\% at strict operating points, and full MLLM embeddings provide substantial advantages over raw ViT features through enhanced semantic structuring from multimodal pretraining.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
