arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.17152cs.SDeess.AS
收藏

基于时序压缩网络的音频时间尺度修改

Audio Time-Scale Modification with Temporal Compressing Networks

Research Center for Information Technology Innovation, Academia Sinica(中央研究院资讯科技创新研究中心);National Cheng Kung University(国立成功大学);National Sun Yat-sen University(国立中山大学)

Ernie Chu, Ju-Ting Chen, Chia-Ping Chen

AI总结:

本研究提出一种基于时序压缩网络的音频时间尺度修改方法,通过将音频编码为可插值的Neuralgram潜在表示实现时域拉伸,经训练的解码器合成高保真音频,性能与现有基准相当,为该领域研究提供新思路。

摘要:

我们提出了一种用于音频信号时间尺度修改的新方法。传统方法依赖分帧技术或短时傅里叶变换在时域拉伸过程中保留频率,与之不同,我们的神经网络模型将原始音频编码为高层潜在表示,命名为Neuralgram(神经图),其中每个向量代表1024个音频采样点。由于具备足够的压缩比,我们能够对Neuralgram进行任意空间插值以实现时域拉伸。最后,经过训练的神经解码器基于拉伸后的Neuralgram表示合成时间尺度修改后的音频采样。编码器和解码器均通过潜在回归损失与对抗损失训练,以获取高保真音频采样。尽管方法简洁,其性能与现有基准相当,为现代时间尺度修改研究开辟了新的可能性。音频采样可访问https://tsmnet-mmasia23.github.io查看。

英文摘要:

We propose a novel approach for time-scale modification of audio signals. Unlike traditional methods that rely on the framing technique or the short-time Fourier transform to preserve the frequency during temporal stretching, our neural network model encodes the raw audio into a high-level latent representation, dubbed Neuralgram, where each vector represents 1024 audio sample points. Due to a sufficient compression ratio, we are able to apply arbitrary spatial interpolation of the Neuralgram to perform temporal stretching. Finally, a learned neural decoder synthesizes the time-scaled audio samples based on the stretched Neuralgram representation. Both the encoder and decoder are trained with latent regression losses and adversarial losses in order to obtain high-fidelity audio samples. Despite its simplicity, our method has comparable performance compared to the existing baselines and opens a new possibility in research into modern time-scale modification. Audio samples can be found at https://tsmnet-mmasia23.github.io

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑