改进的音频片图像同步中重复和跳跃的处理
Improved Handling of Repeats and Jumps in Audio-Sheet Image Synchronization
摘要：
本文研究了在给定音频记录和原始乐谱图像的情况下自动生成钢琴乐谱的问题。虽然以前的作品专注于合成乐谱，其中数据已被清理和预处理，但我们专注于开发一个系统，可以处理来自IMSLP的原始，未处理的乐谱PDF的混乱。我们调查如何以及现有的系统应付真正的扫描乐谱，填充页和不相关的作品或运动，以及由于跳跃和重复的不连续性。我们发现，系统性能的一个重要瓶颈是正确处理跳跃和重复。特别是，我们发现，以前提出的跳跃DTW算法不执行时，跳跃位置是未知的先验鲁棒性。我们提出了一种新的对齐算法称为分层DTW，可以处理跳跃和重复，即使在跳跃位置是未知的。它首先在每个乐谱行上执行特征级别的对齐，然后在片段级别执行第二次对齐。通过在段级别上操作，它能够对关于特定跳跃的可能性的领域知识进行编码。通过精心控制的实验未处理的乐谱PDF从IMSLP，我们表明，层次DTW显着优于跳转DTW在处理各种类型的跳跃。
英文摘要：
This paper studies the problem of automatically generating piano score following videos given an audio recording and raw sheet music images. Whereas previous works focus on synthetic sheet music where the data has been cleaned and preprocessed, we instead focus on developing a system that can cope with the messiness of raw, unprocessed sheet music PDFs from IMSLP. We investigate how well existing systems cope with real scanned sheet music, filler pages and unrelated pieces or movements, and discontinuities due to jumps and repeats. We find that a significant bottleneck in system performance is handling jumps and repeats correctly. In particular, we find that a previously proposed Jump DTW algorithm does not perform robustly when jump locations are unknown a priori. We propose a novel alignment algorithm called Hierarchical DTW that can handle jumps and repeats even when jump locations are not known. It first performs alignment at the feature level on each sheet music line, and then performs a second alignment at the segment level. By operating at the segment level, it is able to encode domain knowledge about how likely a particular jump is. Through carefully controlled experiments on unprocessed sheet music PDFs from IMSLP, we show that Hierarachical DTW significantly outperforms Jump DTW in handling various types of jumps.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
