TMGAN-PLC：基于时间记忆的生成对抗网络
TMGAN-PLC: Audio Packet Loss Concealment using Temporal Memory Generative Adversarial Network
摘要：
分组交换网络中的实时通信在日常通信中得到了广泛的应用，但在实时性受限的情况下，不可避免地会遭受网络延迟和数据丢失。为了解决这些问题，已经开发了音频分组丢失隐藏（PLC）算法，以通过重构丢失的信息来减轻语音传输失败。受限于传输延迟和设备内存，PLC仍然难以使用相对较小的分组缓冲区来完成高质量的语音重构。在本文中，我们提出了一个时间记忆生成对抗网络的音频PLC，被称为TMGAN-PLC，这是由一个新的嵌套UNet生成器和时域/频域鉴别器。具体地说，嵌套UNet和时间特征线性调制的组合精心设计的发生器，以精细地调整帧内信息，并建立帧间的时间依赖性。为了弥补较长丢失突发造成的语音内容缺失，我们采用多级门控矢量量化器来捕获正确的内容并重建接近真实的平滑音频。在PLC Challenge数据集上的大量实验表明，该方法在语音质量、可懂度和PLCMOS方面具有良好的性能。
英文摘要：
Real-time communications in packet-switched networks have become widely used in daily communication, while they inevitably suffer from network delays and data losses in constrained real-time conditions. To solve these problems, audio packet loss concealment (PLC) algorithms have been developed to mitigate voice transmission failures by reconstructing the lost information. Limited by the transmission latency and device memory, it is still intractable for PLC to accomplish high-quality voice reconstruction using a relatively small packet buffer. In this paper, we propose a temporal memory generative adversarial network for audio PLC, dubbed TMGAN-PLC, which is comprised of a novel nested-UNet generator and the time-domain/frequency-domain discriminators. Specifically, a combination of the nested-UNet and temporal feature-wise linear modulation is elaborately devised in the generator to finely adjust the intra-frame information and establish inter-frame temporal dependencies. To complement the missing speech content caused by longer loss bursts, we employ multi-stage gated vector quantizers to capture the correct content and reconstruct the near-real smooth audio. Extensive experiments on the PLC Challenge dataset demonstrate that the proposed method yields promising performance in terms of speech quality, intelligibility, and PLCMOS.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
