This paper presents the Jazz Transformer, a generative model that utilizes a neural sequence model called the Transformer-XL for modeling lead sheets of Jazz music. Moreover, the model endeavors to incorporate structural events present in the Weimar Jazz Database (WJazzD) for inducing structures in the generated music. While we are able to reduce the training loss to a low value, our listening test suggests however a clear gap between the average ratings of the generated and real compositions. We therefore go one step further and conduct a series of computational analysis of the generated compositions from different perspectives. This includes analyzing the statistics of the pitch class, grooving, and chord progression, assessing the structureness of the music with the help of the fitness scape plot, and evaluating the model's understanding of Jazz music through a MIREX-like continuation prediction task. Our work presents in an analytical manner why machine-generated music to date still falls short of the artwork of humanity, and sets some goals for future work on automatic composition to further pursue.
SONY CSL（索尼计算机科学实验室）；Sorbonne Universités（索邦大学）；UPMC Univ Paris 06（巴黎第六大学）
In the context of contemporary monophonic music, expression can be seen as the difference between a musical performance and its symbolic representation, i.e. a musical score. In this paper, we show how Maximum Entropy (MaxEnt) models can be used to generate musical expression in order to mimic a human performance. As a training corpus, we had a professional pianist play about 150 melodies of jazz, pop, and latin jazz. The results show a good predictive power, validating the choice of our model. Additionally, we set up a listening test whose results reveal that on average, people significantly …
Sony Computer Science Laboratory (CSL)（索尼计算机科学实验室）；Dolby Sweden（杜比瑞典）
Jazz guitar solos are improvised melody lines played on one instrument on top of a chordal accompaniment (comping). As the improvisation happens spontaneously, a reference score is non-existent, only a lead sheet. There are situations, however, when one would like to have the original melody lines in the form of notated music, see the Real Book. The motivation is either for the purpose of practice and imitation or for musical analysis. In this work, an automatic transcriber for jazz guitar solos is developed. It resorts to a very intuitive representation of tonal music signals: the pitchgram. …
Decoding Musical Evolution Through Network Science
Department of Computer Science, Sapienza University of Rome（罗马大学计算机科学系）；Department of Computer, Control and Management Engineering, Sapienza University of Rome（罗马大学计算机、控制与管理工程系）；University of Padova, Department of Mathematics（帕多瓦大学数学系）
Institute of Nuclear Physics, Polish Academy of Sciences（波兰科学院核物理研究所）；Faculty of Physics and Applied Computer Science, AGH University of Science and Technology（AGH科技大学物理与应用计算机科学学院）；Faculty of Mathematics and Natural Sciences, University of Rzeszów（热舒夫大学数学与自然科学学院）
In this work we perform a fractal analysis of 160 pieces of music belonging to six different genres. We show that the majority of the pieces reveal characteristics that allow us to classify them as physical processes called the 1/f (pink) noise. However, this is not true for classical music represented here by Frederic Chopin's works and for some jazz pieces that are much more correlated than the pink noise. We also perform a multifractal (MFDFA) analysis of these music pieces. We show that all the pieces reveal multifractal properties. The richest multifractal structures are observed for pop …
Polish-Japanese Institute of Information Technology（波日信息技术学院）；Interdisciplinary Centre for Mathematical and Computational Modelling (ICM), University of Warsaw（华沙大学跨学科数学与计算建模中心（ICM））
In this paper, we first apply random ferns for classification of real music recordings of a jazz band. No initial segmentation of audio data is assumed, i.e., no onset, offset, nor pitch data are needed. The notion of random ferns is described in the paper, to familiarize the reader with this classification algorithm, which was introduced quite recently and applied so far in image recognition tasks. The performance of random ferns is compared with random forests for the same data. The results of experiments are presented in the paper, and conclusions are drawn.
Indian Institute of Technology Kanpur（印度理工学院坎普尔分校）
Music Genres, as a popular meta-data of music, are very useful to organize, explore or search music datasets. Soft music genres are weighted multiple-genre annotations to songs. In this initial work, we propose horizontally stacked bar charts to represent a music dataset annotated by these soft music genres. For this purpose, we take an example of a toy dataset consisting of songs labelled with help of three music genres; Blues, Jazz and Country. We demonstrate how such a stacked bar chart can be used as a slider for user-input in an interface. We implement this by embedding this genre bar in …
Nanjing University（南京大学）；Microsoft Research Asia（微软亚洲研究院）；Peking University（北京大学）
Symbolic music generation aims to generate music scores automatically. A recent trend is to use Transformer or its variants in music generation, which is, however, suboptimal, because the full attention cannot efficiently model the typically long music sequences (e.g., over 10,000 tokens), and the existing models have shortcomings in generating musical repetition structures. In this paper, we propose Museformer, a Transformer with a novel fine- and coarse-grained attention for music generation. Specifically, with the fine-grained attention, a token of a specific bar directly attends to all the…
Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Model
Hong Kong University of Science and Technology（香港科技大学）；University of Science and Technology Beijing（北京科技大学）；Hong Kong Baptist University（香港浸会大学）；Microsoft（微软公司）；Chinese University of Hong Kong（香港中文大学）
针对现有音频 LLM 编解码器因压缩设计导致的语义缺陷问题，提出 X-Codec 方法，通过在 RVQ 前后融入语义特征与损失，显著降低语音合成 WER 并扩展至非语音任务，提升音频生成性能。
We present in this paper a new approach for polyphonic music transcription using evolution strategies (ES). Automatic music transcription is a complex process that still remains an open challenge. Using an audio signal to be transcribed as target for our ES, information needed to generate a MIDI file can be extracted from this latter one. Many techniques presented in the literature at present exist and a few of them have applied evolutionary algorithms to address this problem in the context of considering it as a search space problem. However, ES have never been applied until now. The experime…
