Sushan Konar
In the previous articles of this series, we have discussed the development of musical scales particularly that of the heptatonic scale which forms the basis of Western classical music today. In this last article, we take a look at the basic structure of scales used in Indian classical music and how different `raga's are generated through the simple process of scale shifting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
Raga is a central musical concept in South Asia, especially India, and we investigate connections between Western classical music and Melakarta raga that is a raga in Karnatak (south Indian) classical music, through musical icosahedron. In our previous study, we introduced some kinds of musical icosahedra connecting various musical concepts in Western music: chromatic/whole tone musical icosahedra, Pythagorean/whole tone musical icosahedra, and exceptional musical icosahedra. In this paper, first, we introduce kinds of musical icosahedra that connect the above musical icosahedra through two ki…
We propose a new way of analyzing musical pieces by using the exceptional musical icosahedra where all the major/minor triads are represented by golden triangles or golden gnomons. First, we introduce a concept of the golden neighborhood that characterizes golden triangles/gnomons that neighbor a given golden triangle or gnomon. Then, we investigate a relation between the exceptional musical icosahedra and the neo-Riemannian theory, and find that the golden neighborhoods and the icosahedron symmetry relate any major/minor triad with any major/minor triad. Second, we show how the exceptional mu…
Music information retrieval is currently an active research area that addresses the extraction of musically important information from audio signals, and the applications of such information. The extracted information can be used for search and retrieval of music in recommendation systems, or to aid musicological studies or even in music learning. Sophisticated signal processing techniques are applied to convert low-level acoustic signal properties to musical attributes which are further embedded in a rule-based or statistical classification framework to link with high-level descriptions such …
In the Pacific Coast of Colombia there is a type of marimba called marimba de chonta, which provides the melodic and harmonic contour for traditional music with characteristic chants and dances. The tunings of this marimba are based on the voice of female singers and allows musical practices, as a transposition that preserves relative distances between bars. Here we show that traditional tunings are consistent with isotonic scales, and that they have changed in the last three decades due to the influence of Western music. Specifically, low octaves have changed into just octaves. Additionally, …
Exploring Isolated Musical Notes as Pre-training Data for Predominant Instrument Recognition in Polyphonic Music
针对复调音乐主奏乐器识别难题，提出用单音孤立音符预训练加微调、结合数据增强的端到端方法，在IRMAS数据集上实现性能提升，还构建了参数仅519K的轻量高效模型。
This demo paper introduces partitura, a Python package for handling symbolic musical information. The principal aim of this package is to handle richly structured musical information as conveyed by modern staff music notation. It provides a much wider range of possibilities to deal with music than the more reductive (but very common) piano roll-oriented approach inspired by the MIDI standard. The package is an open source project and is available on GitHub.
Large Language Models: From Notes to Musical Form
针对深度学习音乐生成缺乏大尺度结构的问题，本文改编语言模型提出新方法，能生成2.5分钟且悦耳度与训练音乐相当的作品。
Modelling human perception of musical similarity is critical for the evaluation of generative music systems, musicological research, and many Music Information Retrieval tasks. Although human similarity judgments are the gold standard, computational analysis is often preferable, since results are often easier to reproduce, and computational methods are much more scalable. Moreover, computation based approaches can be calculated quickly and on demand, which is a prerequisite for use with an online system. We propose StyleRank, a method to measure the similarity between a MIDI file and an arbitr…
Adopting a performer-centric perspective, we frequently encounter two statements: "music flows", and "music is life-like". This dissertation builds on top of the two statements above, resulting in an exploration of the role of temporal asymmetry in music (generalizing "music flows") and its relation to the idea of organicism (generalizing "music is life-like"). We focus on two aspects of temporal asymmetry. The first aspect concerns the vastly different epistemic mechanisms with which we obtain knowledge of the past and the future. A particular musical consequence follows: recurrence. The epis…
Music creation involves not only composing the different parts (e.g., melody, chords) of a musical work but also arranging/selecting the instruments to play the different parts. While the former has received increasing attention, the latter has not been much investigated. This paper presents, to the best of our knowledge, the first deep learning models for rearranging music of arbitrary genres. Specifically, we build encoders and decoders that take a piece of polyphonic musical audio as input and predict as output its musical score. We investigate disentanglement techniques such as adversarial…
Musical Word Embedding for Music Tagging and Retrieval
提出音乐词嵌入方法 MWE，通过学习日常与音乐相关词汇并将其集成到音频-词联合表示框架中，结合多原型训练平衡标注与检索性能，在多数据集和任务上证明其优于传统词嵌入。
Loops, seamlessly repeatable musical segments, are a cornerstone of modern music production. Contemporary artists often mix and match various sampled or pre-recorded loops based on musical criteria such as rhythm, harmony and timbral texture to create compositions. Taking such criteria into account, we present LoopNet, a feed-forward generative model for creating loops conditioned on intuitive parameters. We leverage Music Information Retrieval (MIR) models as well as a large collection of public loop samples in our study and use the Wave-U-Net architecture to map control parameters to audio. …
Generation of Musical Timbres using a Text-Guided Diffusion Model
针对文本转音频系统限制音乐创作中人类创造力的问题，本研究提出结合潜在扩散模型和多模态对比学习的系统，以文本描述为条件生成单个音符音频，支持用户指定音色特征，且无需后续相位检索算法。
This technical report gives a detailed, formal description of the features introduced in the paper: Adrien Ycart, Lele Liu, Emmanouil Benetos and Marcus T. Pearce. "Investigating the Perceptual Validity of Evaluation Metrics for Automatic Piano Music Transcription", Transactions of the International Society for Music Information Retrieval (TISMIR), Accepted, 2020.
In this paper we describe our efforts towards the development of live performance computer-based musical instrumentation. Our design criteria include initial ease of use coupled with a long term potential for virtuosity, minimal and low variance latency, and clear and simple strategies for programming the relationship between gesture and musical result. We present custom controllers and unique adaptations of standard gestural interfaces, a programmable connectivity processor, a communications protocol called Open Sound Control (OSC), and a variety of metaphors for musical control. We further d…
The conception and design of new musical interfaces is a multidisciplinary area that tightly relates technology and artistic creation. In this paper, the author first exposes some of the questions he has posed himself during more than a decade experience as a performer, composer, interface and software designer, and educator. Finally, he illustrates these topics with some examples of his work.
Structuring Concept Space with the Musical Circle of Fifths by Utilizing Music Grammar Based Activations
本文提出谐波环形码框架，通过音乐语法激活构建概念空间，实现抽象认知操作。
State-of-the-art music recommendation systems are based on collaborative filtering, which predicts a user's interest from his listening habits and similarities with other users' profiles. These approaches are agnostic to the song content, and therefore face the cold-start problem: they cannot recommend novel songs without listening history. To tackle this issue, content-aware recommendation incorporates information about the songs that can be used for recommending new items. Most methods falling in this category exploit either user-annotated tags, acoustic features or deeply-learned features. …
Musical Form Generation
本文提出一种结构化音乐生成方法，通过条件生成模型创建音乐片段并实现片段间过渡，将高层作曲提示与低层细节生成分离，并利用大型语言模型建议曲式，从而生成任意长度的连贯作品。
Sori: Streamable and Controllable Model for Non-musical Audio to Symbolic Music Transformation
本研究提出NASMT任务，构建Sori模型，结合DAT与带γ的CFG实现非音乐音频到符号化音乐的可控转换，支持实时流式处理且性能优于相关模型。
In the task of generating music, the art factor plays a big role and is a great challenge for AI. Previous work involving adversarial training to produce new music pieces and modeling the compatibility of variety in music (beats, tempo, musical stems) demonstrated great examples of learning this task. Though this was limited to generating mashups or learning features from tempo and key distributions to produce similar patterns. Compound Word Transformer was able to represent music generation task as a sequence generation challenge involving musical events defined by compound words. These music…
Flexible Control in Symbolic Music Generation via Musical Metadata
本研究提出一种以音乐元数据为输入的自回归符号音乐生成模型，通过训练时随机丢弃元数据词元实现灵活控制，经多维度实验及主观测试验证，其在控制性和音乐质量上均优于其他模型。
This paper focuses on the modeling of musical melodies as networks. Notes of a melody can be treated as nodes of a network. Connections are created whenever notes are played in sequence. We analyze some main tracks coming from different music genres, with melodies played using different musical instruments. We find out that the considered networks are, in general, scale free networks and exhibit the small world property. We measure the main metrics and assess whether these networks can be considered as formed by sub-communities. Outcomes confirm that peculiar features of the tracks can be extr…
Musical composition and 2D cellular automata based on music intervals
本文探讨了基于音乐音程的二维细胞自动机在随机音符阵列中的应用，通过复杂系统和人文方法揭示音乐创作规则对生成有序音符模式的重要性。
In this paper, we explore the tokenized representation of musical scores using the Transformer model to automatically generate musical scores. Thus far, sequence models have yielded fruitful results with note-level (MIDI-equivalent) symbolic representations of music. Although the note-level representations can comprise sufficient information to reproduce music aurally, they cannot contain adequate information to represent music visually in terms of notation. Musical scores contain various musical symbols (e.g., clef, key signature, and notes) and attributes (e.g., stem direction, beam, and tie…
Musical genre's classification has been a relevant research topic. The association between music and genres is fundamental for the media industry, which manages musical recommendation systems, and for music streaming services, which may appear classified by genres. In this context, this work presents a feature extraction method for the automatic classification of musical genres, based on complex networks and their topological measurements. The proposed method initially converts the musics into sequences of musical notes and then maps the sequences as complex networks. Topological measurements …
Human usually composes music by organizing elements according to the musical form to express music ideas. However, for neural network-based music generation, it is difficult to do so due to the lack of labelled data on musical form. In this paper, we develop MeloForm, a system that generates melody with musical form using expert systems and neural networks. Specifically, 1) we design an expert system to generate a melody by developing musical elements from motifs to phrases then to sections with repetitions and variations according to pre-given musical form; 2) considering the generated melody…
In this paper, we propose a new compositional tool that will generate a musical outline of speech recorded/provided by the user for use as a musical building block in their compositions. The tool allows any user to use their own speech to generate musical material, while still being able to hear the direct connection between their recorded speech and the resulting music. The tool is built on our proposed pipeline. This pipeline begins with speech-based signal processing, after which some simple musical heuristics are applied, and finally these pre-processed signals are passed through Transform…
SteerMusic: Enhanced Musical Consistency for Zero-shot Text-guided and Personalized Music Editing
SteerMusic通过delta去噪分数和概念令牌实现零射文本引导和个性化音乐编辑，提升音乐一致性与编辑保真度。
In this paper, we present an ongoing research project related to popularizing the mindset of building new digital musical instruments. We developed a physical kit and software intended to provide beginner users with the first grasp on the development process of a digital musical instrument. We expect that, by using the kit and the software, the users could experiment in a short period the various steps in developing a DMI such as physical structure, electronics, programming, mapping, and sound design. Our approach to popularizing the DMI development process is twofold: reducing the cognitive l…
Computational models of music, while providing good descriptions of melodic development, still cannot fully grasp the general structure comprised of repetitions, transpositions, and reuse of melodic material. We present a corpus of strongly structured baroque allemandes, and describe a top-down approach to abstract the shared structure of their musical content using tree representations produced from pairwise differences between the Schenkerian-inspired analyses of each piece, thereby providing a rich hierarchical description of the corpus.
The impact of Music on the mood or emotion of the listener is a well-researched area in human psychology and behavioral science. In Indian classical music, ragas are the melodic structure that defines the various styles and forms of the music. Each raga has been found to evoke a specific emotion in the listener. With the advent of advanced capabilities of audio signal processing and the application of machine learning, the demand for intelligent music classifiers and recommenders has received increased attention, especially in the 'Music as a service' cloud applications. This paper explores a …
The behavior of users of music streaming services is investigated from the point of view of the temporal dimension of individual songs; specifically, the main object of the analysis is the point in time within a song at which users stop listening and start streaming another song ("skip"). The main contribution of this study is the ascertainment of a correlation between the distribution in time of skipping events and the musical structure of songs. It is also shown that such distribution is not only specific to the individual songs, but also independent of the cohort of users and, under station…
Distances on symbolic musical sequences are needed for a variety of applications, from music retrieval to automatic music generation. These musical sequences belong to a given corpus (or style) and it is obvious that a good distance on musical sequences should take this information into account; being able to define a distance ex nihilo which could be applicable to all music styles seems implausible. A distance could also be invariant under some transformations, such as transpositions, so that it can be used as a distance between musical motives rather than musical sequences. However, to our k…
Siamese Residual Neural Network for Musical Shape Evaluation in Piano Performance Assessment
本文提出轻量级孪生残差神经网络（S-ResNN）将音乐形态评估视为分类问题，通过构建包含4116首乐曲的新数据集，验证了该方法在精确率、召回率和F1分数上显著优于多种基准方法。
Real-world Music Plagiarism Detection With Music Segment Transcription System
本文提出结合多种MIR技术的音乐剽窃检测系统，通过音乐片段转录和多特征分析实现现实世界中的剽窃检测，并公开了用于音乐相似性研究的SMP数据集。
Automatic musical accompaniment is where a human musician is accompanied by a computer musician. The computer musician is able to produce musical accompaniment that relates musically to the human performance. The accompaniment should follow the performance using observations of the notes they are playing. This paper describes a complete and detailed construction of a score following and accompanying system using Hidden Markov Models (HMMs). It details how to train a score HMM, how to deal with polyphonic input, how this HMM work when following score, how to build up a musical accompanist. It p…
Analyzing Byte-Pair Encoding on Monophonic and Polyphonic Symbolic Music: A Focus on Musical Phrase Segmentation
本文研究BPE在不同乐器编制及单、多声部符号音乐中的行为，并通过音乐乐句分割实验表明其词汇高度依赖编制、supertokens可捕捉抽象内容，且性能提升随声部与合并次数而变化。
Music Inpainting is the task of filling in missing or lost information in a piece of music. We investigate this task from an interactive music creation perspective. To this end, a novel deep learning-based approach for musical score inpainting is proposed. The designed model takes both past and future musical context into account and is capable of suggesting ways to connect them in a musically meaningful manner. To achieve this, we leverage the representational power of the latent space of a Variational Auto-Encoder and train a Recurrent Neural Network which learns to traverse this latent spac…
Predicting User Intents and Musical Attributes from Music Discovery Conversations
本文研究音乐发现对话中的意图分类，提出结合聊天历史与单轮查询的方法，并同时分类音乐属性，显著提升F1分数且优于Llama 3的零样本和少样本表现。
As digital music production has become mainstream, the selection of appropriate virtual instruments plays a crucial role in determining the quality of music. To search the musical instrument samples or virtual instruments that make one's desired sound, music producers use their ears to listen and compare each instrument sample in their collection, which is time-consuming and inefficient. In this paper, we call this task as Musical Instrument Retrieval and propose a method for retrieving desired musical instruments using reference music mixture as a query. The proposed model consists of the Sin…
This paper describes a series of projects that explore the possibilities of musical expression through the combination of pre-composed, interlocking, modular components. In particular, this paper presents a modular soundtrack recently composed by the author for "Currents of Creativity," a permanent interactive video wall installation at the Pope John Paul II Cultural Center which is slated to open Easter 2001 in Washington, DC.
Estimating Musical Surprisal in Audio
本研究将符号音乐中用自回归模型预测信息含量（IC）衡量惊奇度的方法拓展至音频领域，训练自回归Transformer预测预训练自编码器的压缩音频潜表示，验证了IC的学习效应、段落分布规律及与多类音乐特征的关联，还探究了其对EEG响应的预测能力并开源了代码。
Computers have been used to analyze and create music since they were first introduced in the 1950s and 1960s. Beginning in the late 1990s, the rise of the Internet and large scale platforms for music recommendation and retrieval have made music an increasingly prevalent domain of machine learning and artificial intelligence research. While still nascent, several different approaches have been employed to tackle what may broadly be referred to as "musical intelligence." This article provides a definition of musical intelligence, introduces a taxonomy of its constituent components, and surveys t…
"I made this (sort of)": Negotiating authorship, confronting fraudulence, and exploring new musical spaces with prompt-based AI music generation
本文作者基于两张用前沿提示词AI音乐生成平台创作的专辑的创作经历，借助大语言模型采访展开自我反思，探讨AI音乐创作中的作者身份、音乐身份转变及新音乐空间等问题，并反思LLM介导自我反思的研究方法。
Musical Attention Transformer: Music Generation Using a Music-Specific Attention Model
本文提出了一种音乐特定的注意力模型，通过整合元信息来提升音乐生成的质量，核心方法是将音乐结构和元数据结合，主要贡献是提高了生成音乐的连贯性和多样性。
RoboJam is a machine-learning system for generating music that assists users of a touchscreen music app by performing responses to their short improvisations. This system uses a recurrent artificial neural network to generate sequences of touchscreen interactions and absolute timings, rather than high-level musical notes. To accomplish this, RoboJam's network uses a mixture density layer to predict appropriate touch interaction locations in space and time. In this paper, we describe the design and implementation of RoboJam's network and how it has been integrated into a touchscreen music app. …
Music is an expressive form of communication often used to convey emotion in scenarios where "words are not enough". Part of this information lies in the musical composition where well-defined language exists. However, a significant amount of information is added during a performance as the musician interprets the composition. The performer injects expressiveness into the written score through variations of different musical properties such as dynamics and tempo. In this paper, we describe a model that can learn to perform sheet music. Our research concludes that the generated performances are…
In this article we explore how the different semantics of spectrograms' time and frequency axes can be exploited for musical tempo and key estimation using Convolutional Neural Networks (CNN). By addressing both tasks with the same network architectures ranging from shallow, domain-specific approaches to deep variants with directional filters, we show that axis-aligned architectures perform similarly well as common VGG-style networks developed for computer vision, while being less vulnerable to confounding factors and requiring fewer model parameters.
Musical creativity enabled by nonlinear oscillations of a bubble in water
研究证明水中气泡在编码古典音乐的声压信号下非线性振荡可产生创造性音乐成果，其响应类似电吉他版本，并提出该特性可用于创建模拟人类音乐创造力的物理启发AI系统。
To stretch a music piece to a given length is a common demand in people's daily lives, e.g., in audio-video synchronization and animation production. However, it is not always guaranteed that the stretched music piece is acceptable for general audience since music stretching suffers from people's perceptual artefacts. Over-stretching a music piece will make it uncomfortable for human psychoacoustic hearing. The research on music stretching resistance attempts to estimate the maximum stretchability of music pieces to further avoid over-stretch. It has been observed that musical genres can signi…
The central role of the face in social interaction and non-verbal communication suggests we explore facial action as a means of musical expression. This paper presents the design, implementation, and preliminary studies of a novel system utilizing face detection and optic flow algorithms to associate facial movements with sound synthesis in a topographically specific fashion. We report on our experience with various gesture-to-sound mappings and applications, and describe our preliminary experiments at musical performance using the system.
The music box operad: Random generation of musical phrases from patterns
本文提出multi-patterns概念，将复调音乐短语抽象为operad代数结构，并基于bud generating systems算法随机生成模拟输入风格的新音乐短语。
Musical pieces can be modeled as complex networks. This fosters innovative ways to categorize music, paving the way towards novel applications in multimedia domains, such as music didactics, multimedia entertainment and digital music generation. Clustering these networks through their main metrics allows grouping similar musical tracks. To show the viability of the approach, we provide results on a dataset of guitar solos.
This work present a music dataset named MusicTM-Dataset, which is utilized in improving the representation learning ability of different types of cross-modal retrieval (CMR). Little large music dataset including three modalities is available for learning representations for CMR. To collect a music dataset, we expand the original musical notation to synthesize audio and generated sheet-music image, and build musical notation based sheet-music image, audio clip and syllable-denotation text as fine-grained alignment, such that the MusicTM-Dataset can be exploited to receive shared representation …
Most work on musical score models (a.k.a. musical language models) for music transcription has focused on describing the local sequential dependence of notes in musical scores and failed to capture their global repetitive structure, which can be a useful guide for transcribing music. Focusing on rhythm, we formulate several classes of Bayesian Markov models of musical scores that describe repetitions indirectly using the sparse transition probabilities of notes or note patterns. This enables us to construct piece-specific models for unseen scores with an unfixed repetitive structure and to der…
Analyzing Musical Characteristics of National Anthems in Relation to Global Indices
本文对169国国歌提取音高、速度、节拍等音频特征，并与和平、幸福、自杀率和犯罪率等全球指数作统计相关分析，发现二者可能存在关联。
MMVA: Multimodal Matching Based on Valence and Arousal across Images, Music, and Musical Captions
MMVA通过多模态匹配基于估值和唤醒度，实现跨图像、音乐和音乐字幕的情感内容捕捉，并在估值-唤醒度预测任务中取得最佳性能。
MusER: Musical Element-Based Regularization for Generating Symbolic Music with Emotion
本文提出基于 VQ-VAE 的 MusER 模型，通过潜在空间中的音乐元素正则化解耦不同元素并操纵其改变音乐情感，在客观与主观评价上均优于现有方法。
We have built a music similarity search engine that lets video producers search by listenable music excerpts, as a complement to traditional full-text search. Our system suggests similar sounding track segments in a large music catalog by training a self-supervised convolutional neural network with triplet loss terms and musical transformations. Semi-structured user interviews demonstrate that we can successfully impress professional video producers with the quality of the search experience, and perceived similarities to query tracks averaged 7.8/10 in user testing. We believe this search tool…
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。