长尾自适应流匹配与显式条件一致性引导的精确多模态人脸合成
Long-Tail Adaptive Flow Matching with Explicit Conditional Consistency Guidance for Precise Multimodal Face Synthesis
浏览论文内容
中文总结 AI 辅助
针对多模态人脸合成中语义对齐不佳的问题，提出EC²Face框架，通过显式条件一致性引导和长尾自适应流匹配，在训练时增强监督，推理无额外开销，稀有属性掩码准确率提升29.38%。
中文摘要 AI 辅助
尽管基于扩散的方法已显著提高了多模态人脸合成的可控性，但其语义对齐仍不理想，因为大多数现有方法依赖隐式潜在空间目标来建模去噪变量与多模态条件之间的关系。这种隐式建模往往不足以强制合成人脸与条件输入之间的精确对应，尤其是在长尾语义掩码分布下，稀有属性获得的优化信号较弱。为解决这些局限性，我们提出了EC²Face，一种通过显式语义监督和分布感知优化来改善语义对齐的多模态人脸合成框架。首先，我们引入了显式条件一致性引导（ECCG），通过在像素空间中解码干净潜在变量的近似反向估计，并显式地将合成图像与文本描述和语义掩码对齐，从而施加直接的一致性监督。进一步设计了一种时间动态调制函数，根据反向估计的时间步依赖可靠性来调整监督强度。其次，我们提出了长尾自适应流匹配（LAFM），该机制基于语义属性频率对空间优化信号进行重新加权，并使用归一化权重以在训练期间保持数值稳定性。重要的是，所有附加模块仅在训练期间使用，不引入额外的推理开销。大量实验表明，EC²Face在生成质量和语义对齐方面均持续优于竞争基线，在稀有属性的掩码准确率上实现了29.38%的提升。
英文摘要
Although diffusion-based methods have substantially improved the controllability of multimodal face synthesis, their semantic alignment remains suboptimal because most existing approaches rely on implicit latent-space objectives to model the relationship between denoising variables and multimodal conditions. Such implicit modeling is often insufficient to enforce precise correspondence between synthesized faces and conditional inputs, especially under long-tailed semantic mask distributions where rare attributes receive weak optimization signals. To address these limitations, we propose EC\textsuperscript{2}Face, a multimodal face synthesis framework that improves semantic alignment through explicit semantic supervision and distribution-aware optimization. First, we introduce Explicit Conditional Consistency Guidance (ECCG), which imposes direct consistency supervision in pixel space by decoding an approximate reverse estimate of the clean latent and explicitly aligning the synthesized image with textual descriptions and semantic masks. A temporal dynamic modulation function is further designed to adapt the supervision strength according to the timestep-dependent reliability of reverse estimation. Second, we propose Long-Tail Adaptive Flow Matching (LAFM), which reweights spatial optimization signals based on semantic attribute frequency, with normalized weights to maintain numerical stability during training. Importantly, all additional modules are used only during training and introduce no extra inference overhead. Extensive experiments show that EC\textsuperscript{2}Face consistently outperforms competitive baselines in both generation quality and semantic alignment, achieving a 29.38\% improvement in mask accuracy on rare attributes.
发表机构
- Tsinghua University（清华大学）
- Ant Group（蚂蚁集团）
- Beijing Jiaotong University（北京交通大学）
- South China University of Technology（华南理工大学）
机构由 AI 辅助整理，请以论文原文为准。
