Tempo-Invariant Processing of Rhythm with Convolutional Neural Networks
Anders Elowsson
备注:
Included in doctoral dissertation "Modeling Music: Studies of Music Transcription, Music Perception and Music Production". 26 pages, G5 format. Feedback always welcome
Rhythm patterns can be performed with a wide variation of tempi. This presents a challenge for many music information retrieval (MIR) systems; ideally, perceptually similar rhythms should be represented and processed similarly, regardless of the specific tempo at which they were performed. Several recent systems for tempo estimation, beat tracking, and downbeat tracking have therefore sought to process rhythm in a tempo-invariant way, often by sampling input vectors according to a precomputed pulse level. This paper describes how a log-frequency representation of rhythm-related activations instead can promote tempo invariance when processed with convolutional neural networks. The strategy incorporates invariance at a fundamental level and can be useful for most tasks related to rhythm processing. Different methods are described, relying on magnitude, phase relationships of different rhythm channels, as well as raw phase information. Several variations are explored to provide direction for future implementations.
Dance-to-Music Generation with Encoder-based Textual Inversion
Institute of Automation, CAS(中国科学院自动化研究所);School of Artificial Intelligence, UCAS(中国科学院大学人工智能学院);University of Chinese Academy of Sciences(中国科学院大学);Kuaishou Technology(快手科技);University of Konstanz(康斯坦茨大学);National Cheng-Kung University(成功大学)
Tempo as the Stable Cue: Hierarchical Mixture of Tempo and Beat Experts for Music to 3D Dance Generation
School of Electronic Engineering, Xidian University(西安电子科技大学电子工程学院);Hohai university(河海大学);School of Computer Science and Technology, Xidian University(西安电子科技大学计算机科学与技术学院);Institute for Infocomm Research (I 2 R), A*STAR(新加坡科技研究局信息与通信研究所)
Acoustic characterization of speech rhythm: going beyond metrics with recurrent neural networks
Purdue University(普渡大学);Ghent University(根特大学);Integrative Neuroscience and Cognition Center, CNRS & Université Paris Cité(整合神经科学与认知中心,法国国家科学研究中心与巴黎西岱大学);University of Padua(帕多瓦大学)
Utterance Weighted Multi-Dilation Temporal Convolutional Networks for Monaural Speech Dereverberation
The University of Sheffield(谢菲尔德大学)
语音去混响是许多语音技术应用中的一个重要阶段。最近在这一领域的工作一直由深度神经网络模型主导。时间卷积网络(TCN)是深度学习模型,已被提出用于语音去混响任务中的序列建模。在这项工作中,加权多膨胀深度可分离卷积提出了取代标准的深度可分离卷积TCN模型。这种建议的卷积使TCN能够动态地关注网络中每个卷积块的感受野中或多或少的局部信息。结果表明,这种加权多膨胀时间卷积网络(WD-TCN)在各种模型配置中始终优于TCN,并且使用WD-TCN模型是一种比增加卷积块数量更有效的参数方法来提高模型的性能。与基线TCN相比,最佳性能改善为0.55 dB尺度不变信号失真比(SISDR),而WD-TCN模型在WHAMR数据集上的最佳性能达到12.26 dB SISDR。
R&B -- Rhythm and Brain: Cross-subject Decoding of Music from Human Brain Activity
Department of Biomedicine and Prevention University of Rome Tor Vergata(生物医学与预防系罗马大学托尔维加塔分校);A.A. Martinos Center for Biomedical Imaging Harvard Medical School/MGH, Boston (US)(A.A. Martinos生物医学成像中心哈佛医学院/马萨诸塞总医院,波士顿(美国))