arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.10358eess.AScs.SD
收藏

超低延迟语音增强——一项综合研究

Ultra-Low Latency Speech Enhancement - A Comprehensive Study

National Taiwan University(台湾大学);Microsoft(微软公司)

Haibin Wu, Sebastian Braun

AI总结:

针对现有低延迟语音增强方法缺乏公平对比、多在小型模拟数据测试的问题,本研究基于大规模数据统一训练,综合探究多种低延迟技术及Mamba架构在真实场景下的性能,为超低延迟语音增强研究提供可靠参考。

摘要:

语音增强模型需满足助听设备通常小于5 ms的极低延迟要求。尽管已有多种低延迟技术被提出,但在受控设置下使用深度神经网络(DNNs)对这些方法进行比较的研究仍属空白。过往论文在任务、训练数据、脚本和评估设置上存在差异,无法实现公平对比。此外,所有方法均在小型模拟数据集上测试,难以公平评估其在真实场景下的性能,这可能影响科学结论的可靠性。为解决这些问题,我们在大规模数据上采用一致的训练流程,综合研究多种低延迟技术,并在真实数据上使用更相关的指标进行评估。具体而言,我们探究了非对称窗、可学习窗、自适应时域滤波器组以及未来帧预测技术的有效性。此外,我们还检验了增大模型规模是否能弥补窗尺寸减小带来的影响,以及新型Mamba架构在低延迟环境下的表现。

英文摘要:

Speech enhancement models should meet very low latency requirements typically smaller than 5 ms for hearing assistive devices. While various low-latency techniques have been proposed, comparing these methods in a controlled setup using DNNs remains blank. Previous papers have variations in task, training data, scripts, and evaluation settings, which make fair comparison impossible. Moreover, all methods are tested on small, simulated datasets, making it difficult to fairly assess their performance in real-world conditions, which could impact the reliability of scientific findings. To address these issues, we comprehensively investigate various low-latency techniques using consistent training on large-scale data and evaluate with more relevant metrics on real-world data. Specifically, we explore the effectiveness of asymmetric windows, learnable windows, adaptive time domain filterbanks, and the future-frame prediction technique. Additionally, we examine whether increasing the model size can compensate for the reduced window size, as well as the novel Mamba architecture in low-latency environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑