用于复音乐器计算机伴奏的神经乐谱跟随器
A Neural Score Follower for Computer Accompaniment of Polyphonic Musical Instruments
AI总结：
本文提出基于深度学习的乐谱跟随框架HeurMiT，通过神经潜在表示、实时MIDI数据增强和启发式规则实现复音乐器伴奏中的演奏者跟踪，并分析其局限与未来方向。
摘要：
基于计算机的实时人类音乐表演伴奏涉及三个关键任务：识别表演者正在演奏的内容、定位其在乐谱中的位置，以及同步演奏伴奏声部。其中，第二个任务（乐谱跟随）已通过字符串序列上的动态规划、隐马尔可夫模型（HMMs）和在线时间规整（OLTW）等方法得到解决。然而，深度学习（DL）这一极为成功的技术尚未被直接应用于该问题。因此，我们提出了HeurMiT，一种新颖的基于深度学习的乐谱跟随框架，利用一种神经架构来学习压缩的潜在表示，从而在表演者偏离乐谱的情况下仍能实现精确跟踪。同时，我们实现了一个实时MIDI数据增强工具包，旨在增强这些学习表示的鲁棒性。此外，我们将整个系统与简单的启发式规则相结合，构建了一个能够与现有转录和伴奏技术无缝对接的综合框架。然而，充分的实验表明，尽管HeurMiT具有出色的计算效率，但其内在局限性使其难以在实际乐谱跟随场景中实用。因此，我们将本工作作为对基于深度学习的乐谱跟随器领域的一次初步探索，同时指出一些有前景的方向，以鼓励未来研究朝着鲁棒、最先进的神经乐谱跟随系统发展。
英文摘要：
Real-time computer-based accompaniment for human musical performances entails three critical tasks: identifying what the performer is playing, locating their position within the score, and synchronously playing the accompanying parts. Among these, the second task (score following) has been addressed through methods such as dynamic programming on string sequences, Hidden Markov Models (HMMs), and Online Time Warping (OLTW). Yet, the remarkably successful techniques of Deep Learning (DL) have not been directly applied to this problem. Therefore, we introduce HeurMiT, a novel DL-based score-following framework, utilizing a neural architecture designed to learn compressed latent representations that enables precise performer tracking despite deviations from the score. Parallelly, we implement a real-time MIDI data augmentation toolkit, aimed at enhancing the robustness of these learned representations. Additionally, we integrate the overall system with simple heuristic rules to create a comprehensive framework that can interface seamlessly with existing transcription and accompaniment technologies. However, thorough experimentation reveals that despite its impressive computational efficiency, HeurMiT's underlying limitations prevent it from being practical in real-world score following scenarios. Consequently, we present our work as an introductory exploration into the world of DL-based score followers, while highlighting some promising avenues to encourage future research towards robust, state-of-the-art neural score following systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
