arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2302.04331cs.LGcs.AIcs.SDeess.AS
收藏

短期记忆卷积

Short-Term Memory Convolutions

Samsung R&D Institute Poland(三星波兰研发中心)

Grzegorz Stefański, Krzysztof Arendt, Paweł Daniluk, Bartłomiej Jasik, Artur Szumaczuk

摘要:

时间序列信号的实时处理是许多实际应用中的关键问题。实时处理的思想在音频领域中尤其重要,因为人类对声音的感知对感知信号中的任何类型的干扰都很敏感,特别是听觉和视觉模态之间的滞后。深度学习(DL)模型的兴起使信号处理的前景变得复杂。虽然与标准DSP方法相比,它们通常具有更高的质量,但这一优势因更高的延迟而减弱。在这项工作中,我们提出了一种新的方法,最大限度地减少推理时间延迟和内存消耗,称为短期记忆卷积(STMC)和它的转置对应。STMC的主要优势是与长短期记忆(LSTM)网络相比的低延迟。此外,由于该方法仅基于卷积神经网络(CNN),因此基于STMC的模型的训练更快且更稳定。在这项研究中,我们演示了这种解决方案的应用程序的语音分离任务和GhostNet模型的U-Net模型在声学场景分类(ASC)任务。在语音分离的情况下,我们实现了推理时间减少5倍,延迟减少2倍,而不影响输出质量。ASC任务的推理时间快了4倍,同时保持了原始的准确性。

英文摘要:

The real-time processing of time series signals is a critical issue for many real-life applications. The idea of real-time processing is especially important in audio domain as the human perception of sound is sensitive to any kind of disturbance in perceived signals, especially the lag between auditory and visual modalities. The rise of deep learning (DL) models complicated the landscape of signal processing. Although they often have superior quality compared to standard DSP methods, this advantage is diminished by higher latency. In this work we propose novel method for minimization of inference time latency and memory consumption, called Short-Term Memory Convolution (STMC) and its transposed counterpart. The main advantage of STMC is the low latency comparable to long short-term memory (LSTM) networks. Furthermore, the training of STMC-based models is faster and more stable as the method is based solely on convolutional neural networks (CNNs). In this study we demonstrate an application of this solution to a U-Net model for a speech separation task and GhostNet model in acoustic scene classification (ASC) task. In case of speech separation we achieved a 5-fold reduction in inference time and a 2-fold reduction in latency without affecting the output quality. The inference time for ASC task was up to 4 times faster while preserving the original accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑