发表机构
Yijiahe AI; Tianjin University; Nanjing University（Yijiahe AI; 天津大学; 南京大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出用于真实场景语音分离的解耦式视听增强框架DAVE，构建含219411个混合样本的DAVE-Corpus，采用渐进式多目标优化与选择性增强链，在挑战赛中展现出良好鲁棒性。
AI 中文摘要
真实场景下的视听语音增强仍具挑战性，原因在于视觉输入不可靠且缺乏具备真实声学条件的大规模训练数据。现有方法通常将视觉特征直接融合进分离网络，导致其易受退化视觉信号影响。本文提出DAVE，一种用于真实场景语音分离的解耦式视听增强框架。首先，为解决数据稀缺问题，构建DAVE-Corpus，这一包含219411个混合样本的大规模训练语料，通过对公开会议语料进行组合声学增强生成。随后，引入渐进式多目标优化策略，联合提升语音分离、可懂度、说话人身份保留及感知质量。进一步开发经认证的选择性增强链，仅在无参考划分内应用场景路由、基于GAN的去噪及响度归一化，确保基于参考的指标不会退化。在真实场景视听语音增强挑战赛上的实验表明，DAVE在真实混合场景及视觉退化条件下均具备鲁棒性。
英文摘要
Audio-visual speech enhancement under real-world conditions remains challenging due to unreliable visual inputs and the lack of large-scale training data with realistic acoustic conditions. Existing approaches usually fuse visual features directly into the separation network, making them vulnerable to degraded visual signals. In this paper, we present DAVE, a decoupled audio-visual enhancement framework for real-world speech separation. Firstly, to address the data scarcity issue, we construct DAVE-Corpus, a large-scale training corpus with 219,411 mixtures generated from public meeting corpora through combinatorial acoustic augmentation. Then, we introduce a progressive multi-objective optimization strategy to jointly improve speech separation, intelligibility, speaker identity preservation, and perceptual quality. We further develop a certified selective enhancement chain that applies scene routing, GAN-based denoising, and loudness normalization only within the no-reference partition, guaranteeing non-degradation of reference-based metrics. Experimental results on the Real-World Audio-Visual Speech Enhancement Challenge demonstrate the robustness of DAVE under both real-world mixed scenarios and visual degradation conditions.
