This article investigates a cross-modal retrieval problem in which a user would like to retrieve a passage of music from a MIDI file by taking a cell phone picture of several lines of sheet music. This problem is challenging for two reasons: it has a significant runtime constraint since it is a user-facing application, and there is very little relevant training data containing cell phone images of sheet music. To solve this problem, we introduce a novel feature representation called a bootleg score which encodes the position of noteheads relative to staff lines in sheet music. The MIDI representation can be converted into a bootleg score using deterministic rules of Western musical notation, and the sheet music image can be converted into a bootleg score using classical computer vision techniques for detecting simple geometrical shapes. Once the MIDI and cell phone image have been converted into bootleg scores, we can estimate the alignment using dynamic programming. The most notable characteristic of our system is that it has no trainable weights at all -- only a set of about 40 hyperparameters. With a training set of just 400 images, we show that our system generalizes well to a much larger set of 1600 test images from 160 unseen musical scores. Our system achieves a test F measure score of 0.89, has an average runtime of 0.90 seconds, and outperforms baseline systems based on music object detection and sheet-audio alignment. We provide extensive experimental validation and analysis of our system.
Using a Bi-directional LSTM Model with Attention Mechanism trained on MIDI Data for Generating Unique Music
Just Another Media Laboratory (JAM Lab)（贾斯特另一媒体实验室（JAM实验室））；GITAM University（甘地理工与管理大学）
生成音乐是机器学习领域中一个有趣且具有挑战性的问题。近年来，模仿人类的创造力一直很流行，特别是在计算机视觉和图像处理领域。随着GANs的出现，可以基于训练数据生成新的类似图像。但这对音乐来说就不一样了，因为音乐有一个额外的时间维度。因此，有必要了解音乐是如何以数字形式表现的。当构建执行此生成任务的模型时，学习和生成部分是在一些高级表示中完成的，例如乐器数字接口（Musical Instrument Digital Interface）或乐谱。本文提出了一种具有注意力机制的双向LSTM（长短期记忆）模型，该模型能够基于音乐数据生成相似类型的音乐。由模型生成的音乐遵循模型训练的音乐的主题/风格。此外，由于音乐的性质，节奏，乐器和其他参数可以在生成后定义和更改。
MIDI-LLaMA: An Instruction-Following Multimodal LLM for Symbolic Music Understanding
SensiLab, Monash University, Australia（Monash大学）；University of Sussex, Brighton, United Kingdom（Sussex大学）；School of Computing and Information Systems, The University of Melbourne, Australia（墨尔本大学计算机与信息系统学院）
Emotion4MIDI: a Lyrics-based Emotion-Labeled Symbolic Music Dataset
Institute for Systems and Computer Engineering, Technology and Science (INESC TEC)（系统与计算机工程、技术与科学研究所（INESC TEC））；Faculty of Engineering, University of Porto（波尔图大学工程学院）；ISEP, Polytechnic of Porto, School of Engineering（波尔图理工学院工程学院（ISEP））
