鲁棒音视频语音增强：利用先进后处理纠正复杂环境中的错误分配
Robust Audio-Visual Speech Enhancement: Correcting Misassignments in Complex Environments with Advanced Post-Processing
AI总结：
本文提出后处理分类器PPC并结合mixup训练策略，用于纠正音视频语音增强中因视频质量差和数据不匹配导致的错误输出，在AVSE-challenge数据集上显著超越基线模型。
摘要：
本文针对音视频语音增强（AVSE）系统中普遍存在的错误语音输出问题，该问题通常由视频质量差以及训练与测试数据不匹配引起。我们引入了一种后处理分类器（PPC）来纠正这些错误输出，确保增强后的语音准确对应于目标说话人。我们还在PPC训练中采用了mixup策略以提高其鲁棒性。在AVSE-challenge数据集上的实验结果表明，将PPC集成到AVSE模型中能够显著提升AVSE性能，并且将PPC与使用置换不变训练（PIT）训练的AVSE模型相结合可获得最佳性能。所提出的方法以较大幅度显著优于基线模型。这项工作凸显了该方法在各种模态和架构中更广泛应用的潜力，为该领域未来研究提供了一个有前景的方向。
英文摘要：
This paper addresses the prevalent issue of incorrect speech output in audio-visual speech enhancement (AVSE) systems, which is often caused by poor video quality and mismatched training and test data. We introduce a post-processing classifier (PPC) to rectify these erroneous outputs, ensuring that the enhanced speech corresponds accurately to the intended speaker. We also adopt a mixup strategy in PPC training to improve its robustness. Experimental results on the AVSE-challenge dataset show that integrating PPC into the AVSE model can significantly improve AVSE performance, and combining PPC with the AVSE model trained with permutation invariant training (PIT) yields the best performance. The proposed method substantially outperforms the baseline model by a large margin. This work highlights the potential for broader applications across various modalities and architectures, providing a promising direction for future research in this field.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
