Adrien Ycart, Lele Liu, Emmanouil Benetos, Marcus T. Pearce
This technical report gives a detailed, formal description of the features introduced in the paper: Adrien Ycart, Lele Liu, Emmanouil Benetos and Marcus T. Pearce. "Investigating the Perceptual Validity of Evaluation Metrics for Automatic Piano Music Transcription", Transactions of the International Society for Music Information Retrieval (TISMIR), Accepted, 2020.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
Distances on symbolic musical sequences are needed for a variety of applications, from music retrieval to automatic music generation. These musical sequences belong to a given corpus (or style) and it is obvious that a good distance on musical sequences should take this information into account; being able to define a distance ex nihilo which could be applicable to all music styles seems implausible. A distance could also be invariant under some transformations, such as transpositions, so that it can be used as a distance between musical motives rather than musical sequences. However, to our k…
Musical instrument classification is one of the focuses of Music Information Retrieval (MIR). In order to solve the problem of poor performance of current musical instrument classification models, we propose a musical instrument classification algorithm based on multi-channel feature fusion and XGBoost. Based on audio feature extraction and fusion of the dataset, the features are input into the XGBoost model for training; secondly, we verified the superior performance of the algorithm in the musical instrument classification task by com-paring different feature combinations and several classic…
Analyzing Byte-Pair Encoding on Monophonic and Polyphonic Symbolic Music: A Focus on Musical Phrase Segmentation
本文研究BPE在不同乐器编制及单、多声部符号音乐中的行为，并通过音乐乐句分割实验表明其词汇高度依赖编制、supertokens可捕捉抽象内容，且性能提升随声部与合并次数而变化。
Loops, seamlessly repeatable musical segments, are a cornerstone of modern music production. Contemporary artists often mix and match various sampled or pre-recorded loops based on musical criteria such as rhythm, harmony and timbral texture to create compositions. Taking such criteria into account, we present LoopNet, a feed-forward generative model for creating loops conditioned on intuitive parameters. We leverage Music Information Retrieval (MIR) models as well as a large collection of public loop samples in our study and use the Wave-U-Net architecture to map control parameters to audio. …
Towards Musically Informed Evaluation of Piano Transcription Models
针对现有钢琴转写评估指标无法反映音乐质量、模型在分布外数据性能下降的问题，本研究提出音乐性知情评估指标，并通过实验对比SOTA模型性能，为转写结果的音乐性误差分析提供支撑。
Music information retrieval is currently an active research area that addresses the extraction of musically important information from audio signals, and the applications of such information. The extracted information can be used for search and retrieval of music in recommendation systems, or to aid musicological studies or even in music learning. Sophisticated signal processing techniques are applied to convert low-level acoustic signal properties to musical attributes which are further embedded in a rule-based or statistical classification framework to link with high-level descriptions such …
Musical pieces can be modeled as complex networks. This fosters innovative ways to categorize music, paving the way towards novel applications in multimedia domains, such as music didactics, multimedia entertainment and digital music generation. Clustering these networks through their main metrics allows grouping similar musical tracks. To show the viability of the approach, we provide results on a dataset of guitar solos.
Exploring Isolated Musical Notes as Pre-training Data for Predominant Instrument Recognition in Polyphonic Music
针对复调音乐主奏乐器识别难题，提出用单音孤立音符预训练加微调、结合数据增强的端到端方法，在IRMAS数据集上实现性能提升，还构建了参数仅519K的轻量高效模型。
Large Language Models: From Notes to Musical Form
针对深度学习音乐生成缺乏大尺度结构的问题，本文改编语言模型提出新方法，能生成2.5分钟且悦耳度与训练音乐相当的作品。
Modelling human perception of musical similarity is critical for the evaluation of generative music systems, musicological research, and many Music Information Retrieval tasks. Although human similarity judgments are the gold standard, computational analysis is often preferable, since results are often easier to reproduce, and computational methods are much more scalable. Moreover, computation based approaches can be calculated quickly and on demand, which is a prerequisite for use with an online system. We propose StyleRank, a method to measure the similarity between a MIDI file and an arbitr…
Human usually composes music by organizing elements according to the musical form to express music ideas. However, for neural network-based music generation, it is difficult to do so due to the lack of labelled data on musical form. In this paper, we develop MeloForm, a system that generates melody with musical form using expert systems and neural networks. Specifically, 1) we design an expert system to generate a melody by developing musical elements from motifs to phrases then to sections with repetitions and variations according to pre-given musical form; 2) considering the generated melody…
Sori: Streamable and Controllable Model for Non-musical Audio to Symbolic Music Transformation
本研究提出NASMT任务，构建Sori模型，结合DAT与带γ的CFG实现非音乐音频到符号化音乐的可控转换，支持实时流式处理且性能优于相关模型。
In the task of generating music, the art factor plays a big role and is a great challenge for AI. Previous work involving adversarial training to produce new music pieces and modeling the compatibility of variety in music (beats, tempo, musical stems) demonstrated great examples of learning this task. Though this was limited to generating mashups or learning features from tempo and key distributions to produce similar patterns. Compound Word Transformer was able to represent music generation task as a sequence generation challenge involving musical events defined by compound words. These music…
Flexible Control in Symbolic Music Generation via Musical Metadata
本研究提出一种以音乐元数据为输入的自回归符号音乐生成模型，通过训练时随机丢弃元数据词元实现灵活控制，经多维度实验及主观测试验证，其在控制性和音乐质量上均优于其他模型。
This paper focuses on the modeling of musical melodies as networks. Notes of a melody can be treated as nodes of a network. Connections are created whenever notes are played in sequence. We analyze some main tracks coming from different music genres, with melodies played using different musical instruments. We find out that the considered networks are, in general, scale free networks and exhibit the small world property. We measure the main metrics and assess whether these networks can be considered as formed by sub-communities. Outcomes confirm that peculiar features of the tracks can be extr…
In this paper, we present TIV.lib, an open-source library for the content-based tonal description of musical audio signals. Its main novelty relies on the perceptually-inspired Tonal Interval Vector space based on the Discrete Fourier transform, from which multiple instantaneous and global representations, descriptors and metrics are computed - e.g., harmonic change, dissonance, diatonicity, and musical key. The library is cross-platform, implemented in Python and the graphical programming language Pure Data, and can be used in both online and offline scenarios. Of note is its potential for en…
Deep learning-based approaches to musical source separation are often limited to the instrument classes that the models are trained on and do not generalize to separate unseen instruments. To address this, we propose a few-shot musical source separation paradigm. We condition a generic U-Net source separation model using few audio examples of the target instrument. We train a few-shot conditioning encoder jointly with the U-Net to encode the audio examples into a conditioning vector to configure the U-Net via feature-wise linear modulation (FiLM). We evaluate the trained models on real musical…
The task of classifying emotions within a musical track has received widespread attention within the Music Information Retrieval (MIR) community. Music emotion recognition has traditionally relied on the use of acoustic features, verbal features, and metadata-based filtering. The role of musical prosody remains under-explored despite several studies demonstrating a strong connection between prosody and emotion. In this study, we restrict the input of traditional machine learning algorithms to the features of musical prosody. Furthermore, our proposed approach builds upon the prior by classifyi…
Aligning Language Models for Lyric-to-Melody Generation with Rule-Based Musical Constraints
本文提出基于规则的音乐约束对齐框架，通过DPO和KTO优化减少旋律生成中的约束违规，提升音乐性和连贯性。
Most work on musical score models (a.k.a. musical language models) for music transcription has focused on describing the local sequential dependence of notes in musical scores and failed to capture their global repetitive structure, which can be a useful guide for transcribing music. Focusing on rhythm, we formulate several classes of Bayesian Markov models of musical scores that describe repetitions indirectly using the sparse transition probabilities of notes or note patterns. This enables us to construct piece-specific models for unseen scores with an unfixed repetitive structure and to der…
Musical Attention Transformer: Music Generation Using a Music-Specific Attention Model
本文提出了一种音乐特定的注意力模型，通过整合元信息来提升音乐生成的质量，核心方法是将音乐结构和元数据结合，主要贡献是提高了生成音乐的连贯性和多样性。
Real-world Music Plagiarism Detection With Music Segment Transcription System
本文提出结合多种MIR技术的音乐剽窃检测系统，通过音乐片段转录和多特征分析实现现实世界中的剽窃检测，并公开了用于音乐相似性研究的SMP数据集。
RoboJam is a machine-learning system for generating music that assists users of a touchscreen music app by performing responses to their short improvisations. This system uses a recurrent artificial neural network to generate sequences of touchscreen interactions and absolute timings, rather than high-level musical notes. To accomplish this, RoboJam's network uses a mixture density layer to predict appropriate touch interaction locations in space and time. In this paper, we describe the design and implementation of RoboJam's network and how it has been integrated into a touchscreen music app. …
Musical Instrument Identification has for long had a reputation of being one of the most ill-posed problems in the field of Musical Information Retrieval(MIR). Despite several robust attempts to solve the problem, a timeline spanning over the last five odd decades, the problem remains an open conundrum. In this work, the authors take on a further complex version of the traditional problem statement. They attempt to solve the problem with minimal data available - one audio excerpt per class. We propose to use a convolutional Siamese network and a residual variant of the same to identify musical…
This work aims to examine one of the cornerstone problems of Musical Instrument Retrieval (MIR), in particular, instrument classification. IRMAS (Instrument recognition in Musical Audio Signals) data set is chosen for this purpose. The data includes musical clips recorded from various sources in the last century, thus having a wide variety of audio quality. We have presented a very concise summary of past work in this domain. Having implemented various supervised learning algorithms for this classification task, SVM classifier has outperformed the other state-of-the-art models with an accuracy…
Estimating Musical Surprisal in Audio
本研究将符号音乐中用自回归模型预测信息含量（IC）衡量惊奇度的方法拓展至音频领域，训练自回归Transformer预测预训练自编码器的压缩音频潜表示，验证了IC的学习效应、段落分布规律及与多类音乐特征的关联，还探究了其对EEG响应的预测能力并开源了代码。
"I made this (sort of)": Negotiating authorship, confronting fraudulence, and exploring new musical spaces with prompt-based AI music generation
本文作者基于两张用前沿提示词AI音乐生成平台创作的专辑的创作经历，借助大语言模型采访展开自我反思，探讨AI音乐创作中的作者身份、音乐身份转变及新音乐空间等问题，并反思LLM介导自我反思的研究方法。
MusER: Musical Element-Based Regularization for Generating Symbolic Music with Emotion
本文提出基于 VQ-VAE 的 MusER 模型，通过潜在空间中的音乐元素正则化解耦不同元素并操纵其改变音乐情感，在客观与主观评价上均优于现有方法。
Musical Voice Separation as Link Prediction: Modeling a Musical Perception Task as a Multi-Trajectory Tracking Problem
该研究将复调音乐的声部分离任务建模为多轨迹跟踪问题，通过异构图神经网络预测音符间链路实现分离，提出新正则化损失约束轨迹结构，在多风格古典音乐任务中达到新的最优水平。
Opening the Design Space: Two Years of Performance with Intelligent Musical Instruments
本文探讨了在可访问且便携的AI系统支持下，智能音乐乐器的设计空间，通过两年的艺术研究，展示了AI交互的新策略和方法。
Analysis and Visualization of Musical Structure using Networks
本文提出了一个从符号音乐信息构建图或网络家族的框架，通过图论方法分析数字乐谱数据以可视化音乐特征并理解其结构元素，验证了该方法在多风格音乐分析中的有效性。
MusiChat: Vibe Composing for Music Creation
研究针对AI音乐创作中迭代难问题，提出MusiChat系统，通过分层可控框架、记忆增强架构及混合意图路由机制，实现人机协作音乐创作，经评估在交互准确率和用户喜好比例上表现良好，支持多轮音乐创作和人机共创。
We present a novel framework for generating pop music. Our model is a hierarchical Recurrent Neural Network, where the layers and the structure of the hierarchy encode our prior knowledge about how pop music is composed. In particular, the bottom layers generate the melody, while the higher levels produce the drums and chords. We conduct several human studies that show strong preference of our generated music over that produced by the recent method by Google. We additionally show two applications of our framework: neural dancing and karaoke, as well as neural story singing.
Notes in a musical piece are building blocks employed in non-random ways to create melodies. It is the "interaction" among a limited amount of notes that allows constructing the variety of musical compositions that have been written in centuries and within different cultures. Networks are a modeling tool that is commonly employed to represent a set of entities interacting in some way. Thus, notes composing a melody can be seen as nodes of a network that are connected whenever these are played in sequence. The outcome of such a process results in a directed graph. By using complex network theor…
We apply deep learning methods, specifically long short-term memory (LSTM) networks, to music transcription modelling and composition. We build and train LSTM networks using approximately 23,000 music transcriptions expressed with a high-level vocabulary (ABC notation), and use them to generate new transcriptions. Our practical aim is to create music transcription models useful in particular contexts of music composition. We present results from three perspectives: 1) at the population level, comparing descriptive statistics of the set of training transcriptions and generated transcriptions; 2…
Apollo: An Interactive Environment for Generating Symbolic Musical Phrases using Corpus-based Style Imitation
Apollo是一个交互式桌面应用，利用基于语料库的风格模仿技术生成西方音乐符号短语，支持语料库构建与管理，并可导出MIDI格式音乐材料。
MuScriptor: An Open Model for Multi-Instrument Music Transcription
研究多乐器音乐转录问题，核心方法是分析合成数据预训练有效性，结合真实音乐音频微调与强化学习后训练，还引入乐器存在条件定制转录，主要贡献是发布MuScriptor开放模型用于多种音乐录音。
Raga is a central musical concept in South Asia, especially India, and we investigate connections between Western classical music and Melakarta raga that is a raga in Karnatak (south Indian) classical music, through musical icosahedron. In our previous study, we introduced some kinds of musical icosahedra connecting various musical concepts in Western music: chromatic/whole tone musical icosahedra, Pythagorean/whole tone musical icosahedra, and exceptional musical icosahedra. In this paper, first, we introduce kinds of musical icosahedra that connect the above musical icosahedra through two ki…
State-of-the-art music recommendation systems are based on collaborative filtering, which predicts a user's interest from his listening habits and similarities with other users' profiles. These approaches are agnostic to the song content, and therefore face the cold-start problem: they cannot recommend novel songs without listening history. To tackle this issue, content-aware recommendation incorporates information about the songs that can be used for recommending new items. Most methods falling in this category exploit either user-annotated tags, acoustic features or deeply-learned features. …
The original MV2H metric was designed to evaluate systems which transcribe from an input audio (or MIDI) piece to a complete musical score. However, it requires both the transcribed score and the ground truth score to be time-aligned with the input. Some recent work has begun to transcribe directly from an audio signal into a musical score, skipping the alignment step. This paper introduces an automatic alignment method based on dynamic time warp which allows for MV2H to be used to evaluate such non-aligned transcriptions. This has the additional benefit of allowing non-aligned musical scores-…
The behavior of users of music streaming services is investigated from the point of view of the temporal dimension of individual songs; specifically, the main object of the analysis is the point in time within a song at which users stop listening and start streaming another song ("skip"). The main contribution of this study is the ascertainment of a correlation between the distribution in time of skipping events and the musical structure of songs. It is also shown that such distribution is not only specific to the individual songs, but also independent of the cohort of users and, under station…
We propose a new way of analyzing musical pieces by using the exceptional musical icosahedra where all the major/minor triads are represented by golden triangles or golden gnomons. First, we introduce a concept of the golden neighborhood that characterizes golden triangles/gnomons that neighbor a given golden triangle or gnomon. Then, we investigate a relation between the exceptional musical icosahedra and the neo-Riemannian theory, and find that the golden neighborhoods and the icosahedron symmetry relate any major/minor triad with any major/minor triad. Second, we show how the exceptional mu…
In the previous articles of this series, we have discussed the development of musical scales particularly that of the heptatonic scale which forms the basis of Western classical music today. In this last article, we take a look at the basic structure of scales used in Indian classical music and how different `raga's are generated through the simple process of scale shifting.
In the Pacific Coast of Colombia there is a type of marimba called marimba de chonta, which provides the melodic and harmonic contour for traditional music with characteristic chants and dances. The tunings of this marimba are based on the voice of female singers and allows musical practices, as a transposition that preserves relative distances between bars. Here we show that traditional tunings are consistent with isotonic scales, and that they have changed in the last three decades due to the influence of Western music. Specifically, low octaves have changed into just octaves. Additionally, …
Analyzing Musical Characteristics of National Anthems in Relation to Global Indices
本文对169国国歌提取音高、速度、节拍等音频特征，并与和平、幸福、自杀率和犯罪率等全球指数作统计相关分析，发现二者可能存在关联。
This demo paper introduces partitura, a Python package for handling symbolic musical information. The principal aim of this package is to handle richly structured musical information as conveyed by modern staff music notation. It provides a much wider range of possibilities to deal with music than the more reductive (but very common) piano roll-oriented approach inspired by the MIDI standard. The package is an open source project and is available on GitHub.
This work present a music dataset named MusicTM-Dataset, which is utilized in improving the representation learning ability of different types of cross-modal retrieval (CMR). Little large music dataset including three modalities is available for learning representations for CMR. To collect a music dataset, we expand the original musical notation to synthesize audio and generated sheet-music image, and build musical notation based sheet-music image, audio clip and syllable-denotation text as fine-grained alignment, such that the MusicTM-Dataset can be exploited to receive shared representation …
Adopting a performer-centric perspective, we frequently encounter two statements: "music flows", and "music is life-like". This dissertation builds on top of the two statements above, resulting in an exploration of the role of temporal asymmetry in music (generalizing "music flows") and its relation to the idea of organicism (generalizing "music is life-like"). We focus on two aspects of temporal asymmetry. The first aspect concerns the vastly different epistemic mechanisms with which we obtain knowledge of the past and the future. A particular musical consequence follows: recurrence. The epis…
This work proposes a novel feature selection algorithm to classify Songs into different groups. Classification of musical content is often a non-trivial job and still relatively less explored area. The main idea conveyed in this article is to come up with a new feature selection scheme that does the classification job elegantly and with high accuracy but with simpler but wisely chosen small number of features thus being less prone to over-fitting. This uses a very basic general idea about the structure of the audio signal which is generally in the shape of a trapezium. So, using this general i…
The conception and design of new musical interfaces is a multidisciplinary area that tightly relates technology and artistic creation. In this paper, the author first exposes some of the questions he has posed himself during more than a decade experience as a performer, composer, interface and software designer, and educator. Finally, he illustrates these topics with some examples of his work.
Musical composition and 2D cellular automata based on music intervals
本文探讨了基于音乐音程的二维细胞自动机在随机音符阵列中的应用，通过复杂系统和人文方法揭示音乐创作规则对生成有序音符模式的重要性。
Structuring Concept Space with the Musical Circle of Fifths by Utilizing Music Grammar Based Activations
本文提出谐波环形码框架，通过音乐语法激活构建概念空间，实现抽象认知操作。
Musical Form Generation
本文提出一种结构化音乐生成方法，通过条件生成模型创建音乐片段并实现片段间过渡，将高层作曲提示与低层细节生成分离，并利用大型语言模型建议曲式，从而生成任意长度的连贯作品。
Music creation involves not only composing the different parts (e.g., melody, chords) of a musical work but also arranging/selecting the instruments to play the different parts. While the former has received increasing attention, the latter has not been much investigated. This paper presents, to the best of our knowledge, the first deep learning models for rearranging music of arbitrary genres. Specifically, we build encoders and decoders that take a piece of polyphonic musical audio as input and predict as output its musical score. We investigate disentanglement techniques such as adversarial…
This paper reports on work in progress on a finger-based tactile I/O device for musical interaction. Central to the device is the ability to set up cyclical relationships between tactile input and output. A direct practical application of this to musical interaction is given, using the idea to multiplex two degrees of freedom on a single tactile loop.
In this paper, we explore the tokenized representation of musical scores using the Transformer model to automatically generate musical scores. Thus far, sequence models have yielded fruitful results with note-level (MIDI-equivalent) symbolic representations of music. Although the note-level representations can comprise sufficient information to reproduce music aurally, they cannot contain adequate information to represent music visually in terms of notation. Musical scores contain various musical symbols (e.g., clef, key signature, and notes) and attributes (e.g., stem direction, beam, and tie…
Musical Word Embedding for Music Tagging and Retrieval
提出音乐词嵌入方法 MWE，通过学习日常与音乐相关词汇并将其集成到音频-词联合表示框架中，结合多原型训练平衡标注与检索性能，在多数据集和任务上证明其优于传统词嵌入。
Musical genre's classification has been a relevant research topic. The association between music and genres is fundamental for the media industry, which manages musical recommendation systems, and for music streaming services, which may appear classified by genres. In this context, this work presents a feature extraction method for the automatic classification of musical genres, based on complex networks and their topological measurements. The proposed method initially converts the musics into sequences of musical notes and then maps the sequences as complex networks. Topological measurements …
Background music affects lyrics intelligibility of singing vocals in a music piece. Automatic lyrics alignment and transcription in polyphonic music are challenging tasks because the singing vocals are corrupted by the background music. In this work, we propose to learn music genre-specific characteristics to train polyphonic acoustic models. We first compare several automatic speech recognition pipelines for the application of lyrics transcription. We then present the lyrics alignment and transcription performance of music-informed acoustic models for the best-performing pipeline, and systema…
We present an automatic piano transcription system that converts polyphonic audio recordings into musical scores. This has been a long-standing problem of music information processing, and recent studies have made remarkable progress in the two main component techniques: multipitch detection and rhythm quantization. Given this situation, we study a method integrating deep-neural-network-based multipitch detection and statistical-model-based rhythm quantization. In the first part, we conducted systematic evaluations and found that while the present method achieved high transcription accuracies …
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。