A cover song, by definition, is a new performance or recording of a previously recorded, commercially released song. It may be by the original artist themselves or a different artist altogether and can vary from the original in unpredictable ways including key, arrangement, instrumentation, timbre and more. In this work we propose a novel approach to learning audio representations for the task of cover song detection. We train a neural architecture on tens of thousands of cover-song audio clips and test it on a held out set. We obtain a mean precision@1 of 65% over mini-batches, ten times better than random guessing. Our results indicate that Siamese network configurations show promise for approaching the cover song identification problem.
We explore the feasibility of using triplet neural networks to embed songs based on content-based music similarity. Our network is trained using triplets of songs such that two songs by the same artist are embedded closer to one another than to a third song by a different artist. We compare two models that are trained using different ways of picking this third song: at random vs. based on shared genre labels. Our experiments are conducted using songs from the Free Music Archive and use standard audio features. The initial results show that shallow Siamese networks can be used to embed music fo…
In this paper, we propose a new approach to cover song identification using a CNN (convolutional neural network). Most previous studies extract the feature vectors that characterize the cover song relation from a pair of songs and used it to compute the (dis)similarity between the two songs. Based on the observation that there is a meaningful pattern between cover songs and that this can be learned, we have reformulated the cover song identification problem in a machine learning framework. To do this, we first build the CNN using as an input a cross-similarity matrix generated from a pair of s…
In this work, we pose and address the following "cover song analogies" problem: given a song A by artist 1 and a cover song A' of this song by artist 2, and given a different song B by artist 1, synthesize a song B' which is a cover of B in the style of artist 2. Normally, such a polyphonic style transfer problem would be quite challenging, but we show how the cover songs example constrains the problem, making it easier to solve. First, we extract the longest common beat-synchronous subsequence between A and A', and we time stretch the corresponding beat intervals in A' so that they align with…
Cover song identification represents a challenging task in the field of Music Information Retrieval (MIR) due to complex musical variations between query tracks and cover versions. Previous works typically utilize hand-crafted features and alignment algorithms for the task. More recently, further breakthroughs are achieved employing neural network approaches. In this paper, we propose a novel Convolutional Neural Network (CNN) architecture based on the characteristics of the cover song task. We first train the network through classification strategies; the network is then used to extract music…
Bytedance AI Lab（字节跳动人工智能实验室）；Peking University（北京大学）
We present in this paper ByteCover, which is a new feature learning method for cover song identification (CSI). ByteCover is built based on the classical ResNet model, and two major improvements are designed to further enhance the capability of the model for CSI. In the first improvement, we introduce the integration of instance normalization (IN) and batch normalization (BN) to build IBN blocks, which are major components of our ResNet-IBN model. With the help of the IBN blocks, our CSI model can learn features that are invariant to the changes of musical attributes such as key, tempo, timbre…
SongBsAb: A Dual Prevention Approach against Singing Voice Conversion based Illegal Song Covers
Pengcheng Laboratory（鹏城实验室）；National University of Singapore（新加坡国立大学）；Nanjing Institute of Software Technology（南京软件技术研究院）；Stony Brook University（石溪大学）；Monash University（莫纳什大学）；Nanyang Technological University（南洋理工大学）
Audio-based cover song detection has received much attention in the MIR community in the recent years. To date, the most popular formulation of the problem has been to compare the audio signals of two tracks and to make a binary decision based on this information only. However, leveraging additional signals might be key if one wants to solve the problem at an industrial scale. In this paper, we introduce an ensemble-based method that approaches the problem from a many-to-many perspective. Instead of considering pairs of tracks in isolation, we consider larger sets of potential versions for a g…
ByteDance（字节跳动）；Zhejiang University（浙江大学）；University of Oxford（牛津大学）
Deep learning based methods have become a paradigm for cover song identification (CSI) in recent years, where the ByteCover systems have achieved state-of-the-art results on all the mainstream datasets of CSI. However, with the burgeon of short videos, many real-world applications require matching short music excerpts to full-length music tracks in the database, which is still under-explored and waiting for an industrial-level solution. In this paper, we upgrade the previous ByteCover systems to ByteCover3 that utilizes local features to further improve the identification performance of short …
Etude: Piano Cover Generation with a Three-Stage Approach -- Extract, strucTUralize, and DEcode
Dept. of Information Management, National Taiwan University, Taipei, Taiwan（信息管理系，国立台湾大学，台北，台湾）；National Taiwan University, Taipei, Taiwan（国立台湾大学，台北，台湾）；Dept. of Information Management National Taiwan University Taipei, Taiwan（信息管理系，国立台湾大学，台北，台湾）
Over 60,000 songs are released on Spotify every day, and the competition for the listener's attention is immense. In that regard, the importance of captivating and inviting cover art cannot be underestimated, because it is deeply entangled with a song's character and the artist's identity, and remains one of the most important gateways to lead people to discover music. However, designing cover art is a highly creative, lengthy and sometimes expensive process that can be daunting, especially for non-professional artists. For this reason, we propose a novel deep-learning framework to generate co…
YuE2: Unifying Symbolic and Audio Music Generation at Frontier Quality
HKUST（香港科技大学）；M-A-P（M-A-P（多模态艺术投影））；New York University（纽约大学）；Stanford University（斯坦福大学）；MBZUAI（穆罕默德·本·扎耶德人工智能大学）；ACE Studio（ACE工作室）；HKGAI（香港生成式人工智能研究院）
Functional embeddings enable Aggregation of multi-area SEEG recordings over subjects and sessions
Department of Biomedical Engineering, University of California Irvine（生物医学工程系，加州大学尔湾分校）；Department of Electrical Engineering, University of California Irvine（电气工程系，加州大学尔湾分校）；Department of Neurology, Rady Children’s Health（神经病学系，Rady儿童健康中心）
Center for Digital Safety & Security（数字安全与安全中心）；Musimap；Faculty of Informatics（信息学院）；Austrian Institute of Technology（奥地利技术研究所）；TU Wien（维也纳技术大学）
Deep Learning has become state of the art in visual computing and continuously emerges into the Music Information Retrieval (MIR) and audio retrieval domain. In order to bring attention to this topic we propose an introductory tutorial on deep learning for MIR. Besides a general introduction to neural networks, the proposed tutorial covers a wide range of MIR relevant deep learning approaches. \textbf{Convolutional Neural Networks} are currently a de-facto standard for deep learning based audio retrieval. \textbf{Recurrent Neural Networks} have proven to be effective in onset detection tasks s…
