AVFSNet：面向灵活说话人数量的音视频语音分离，结合多尺度与多任务学习
AVFSNet: Audio-Visual Speech Separation for Flexible Number of Speakers with Multi-Scale and Multi-Task Learning
AI总结：
本文提出AVFSNet，通过多尺度编码、并行架构及音视频多任务学习，在未知说话人数量条件下实现鲁棒的语音分离与说话人计数。
摘要：
从包含灵活说话人数量的混合信号中分离目标语音是一项具有挑战性的任务。现有方法虽然展现出较强的分离性能和噪声鲁棒性，但主要假设混合信号中的说话人数量已知。针对未知说话人数量场景的研究有限，且在实际声学环境中的泛化能力明显受限。为克服这些挑战，本文提出AVFSNet——一种融合多尺度编码与并行架构的音视频语音分离模型，并联合优化说话人计数与多说话人分离任务。该模型以并行方式独立分离每个说话人，同时通过视觉信息集成增强对环境噪声的适应性。综合实验评估表明，AVFSNet在多个评价指标上取得了最先进的结果，并在多个数据集上表现出卓越的性能。
英文摘要：
Separating target speech from mixed signals containing flexible speaker quantities presents a challenging task. While existing methods demonstrate strong separation performance and noise robustness, they predominantly assume prior knowledge of speaker counts in mixtures. The limited research addressing unknown speaker quantity scenarios exhibits significantly constrained generalization capabilities in real acoustic environments. To overcome these challenges, this paper proposes AVFSNet -- an audio-visual speech separation model integrating multi-scale encoding and parallel architecture -- jointly optimized for speaker counting and multi-speaker separation tasks. The model independently separates each speaker in parallel while enhancing environmental noise adaptability through visual information integration. Comprehensive experimental evaluations demonstrate that AVFSNet achieves state-of-the-art results across multiple evaluation metrics and delivers outstanding performance on diverse datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
