arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2102.09763cs.SDeess.AS
收藏

基于频时注意网络的歌唱旋律提取

Frequency-Temporal Attention Network for Singing Melody Extraction

Fudan University(复旦大学);National Institute of Informatics (NII)(国立情报学研究所);Shanghai Key Laboratory of Intelligent Information Processing, Fudan University(复旦大学上海市智能信息处理重点实验室)

Shuai Yu, Xiaoheng Sun, Yi Yu, Wei Li

摘要:

音乐音频一般由三个物理属性组成:频率、时间和幅度。有趣的是,人类的听觉外围也为这些维度中的每一个提供神经代码来感知音乐。受这些内在特征的启发,提出了一种模仿人类听觉的频时注意网络,用于歌唱旋律提取。特别地,该模型包含频率-时间注意模块和与这三个物理特性相对应的选择性融合模块。频率注意模块用于选择与耳蜗相同的激活频带,时间注意模块负责分析时间模式。最后,选择性融合模块建议重新校准的幅度和融合的原始信息进行预测。此外,我们建议使用另一个分支来同时预测歌声旋律的存在。实验结果表明,该模型优于现有的国家的最先进的方法。

英文摘要:

Musical audio is generally composed of three physical properties: frequency, time and magnitude. Interestingly, human auditory periphery also provides neural codes for each of these dimensions to perceive music. Inspired by these intrinsic characteristics, a frequency-temporal attention network is proposed to mimic human auditory for singing melody extraction. In particular, the proposed model contains frequency-temporal attention modules and a selective fusion module corresponding to these three physical properties. The frequency attention module is used to select the same activation frequency bands as did in cochlear and the temporal attention module is responsible for analyzing temporal patterns. Finally, the selective fusion module is suggested to recalibrate magnitudes and fuse the raw information for prediction. In addition, we propose to use another branch to simultaneously predict the presence of singing voice melody. The experimental results show that the proposed model outperforms existing state-of-the-art methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑