面向可听戴设备亚毫秒级延迟实时语音增强模型的研究
Towards Sub-millisecond Latency Real-Time Speech Enhancement Models on Hearables
AI总结：
本文提出一种基于最小相位FIR滤波器和轻量级LSTM的可听戴设备语音增强方法，实现0.32-1.25毫秒算法延迟和3.35毫秒端到端延迟，单麦克风下SI-SDRi达4.1 dB。
摘要：
低延迟模型对于助听器和可听戴设备等实时语音增强应用至关重要。然而，针对资源受限可听戴设备的亚毫秒级延迟空间仍未得到充分探索。我们展示了一种使用计算高效的最小相位FIR滤波器的语音增强方法，该方法支持逐样本处理，实现了0.32毫秒至1.25毫秒的平均算法延迟。在单麦克风条件下，我们观察到平均SI-SDRi为4.1 dB。该方法在未见过的音频录音上表现出泛化能力，DNSMOS提升了0.2。我们使用一个包含626k参数的轻量级LSTM模型来生成FIR滤波器系数。通过在低功耗DSP上的真实硬件实现，我们的系统能够以376 MIPS运行，并实现3.35毫秒的平均端到端延迟。此外，我们还与现有的低延迟频谱掩蔽技术进行了比较。我们希望这项工作能够促进对延迟的更好理解，并可用于提升可听戴设备的舒适性和可用性。
英文摘要：
Low latency models are critical for real-time speech enhancement applications, such as hearing aids and hearables. However, the sub-millisecond latency space for resource-constrained hearables remains underexplored. We demonstrate speech enhancement using a computationally efficient minimum-phase FIR filter, enabling sample-by-sample processing to achieve mean algorithmic latency of 0.32 ms to 1.25 ms. With a single microphone, we observe a mean SI-SDRi of 4.1 dB. The approach shows generalization with a DNSMOS increase of 0.2 on unseen audio recordings. We use a lightweight LSTM-based model of 626k parameters to generate FIR taps. Using a real hardware implementation on a low-power DSP, our system can run with 376 MIPS and a mean end-to-end latency of 3.35 ms. In addition, we provide a comparison with existing low-latency spectral masking techniques. We hope this work will enable a better understanding of latency and can be used to improve the comfort and usability of hearables.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
