用于多模态钢琴演奏数据集采集与指法标注的两个网络工具包
Two Web Toolkits for Multimodal Piano Performance Dataset Acquisition and Fingering Annotation
- KAIST（韩国科学技术院）
- Georgia Institute of Technology（佐治亚理工学院）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文提出包含PiaRec和ASDF两个图形界面的集成网络工具包，分别用于同步采集音视频、MIDI及元数据并高效标注指法，以解决多模态钢琴演奏数据获取繁琐的瓶颈问题。
AI中文摘要：
钢琴演奏是一种多模态活动，本质上将身体动作与声学演绎结合在一起。尽管对分析钢琴演奏多模态性质的研究兴趣日益增长，但获取大规模多模态数据的繁琐过程仍然是一个重大瓶颈，阻碍了该领域的进一步发展。为了克服这一障碍，我们提出了一个集成式网络工具包，包含两个图形用户界面（GUI）：(i) PiaRec，支持音频、视频、MIDI和演奏元数据的同步采集。(ii) ASDF，能够从视觉数据中高效标注演奏者的指法。总体而言，该系统可以简化多模态钢琴演奏数据集的采集流程。
英文摘要：
Piano performance is a multimodal activity that intrinsically combines physical actions with the acoustic rendition. Despite growing research interest in analyzing the multimodal nature of piano performance, the laborious process of acquiring large-scale multimodal data remains a significant bottleneck, hindering further progress in this field. To overcome this barrier, we present an integrated web toolkit comprising two graphical user interfaces (GUIs): (i) PiaRec, which supports the synchronized acquisition of audio, video, MIDI, and performance metadata. (ii) ASDF, which enables the efficient annotation of performer fingering from the visual data. Collectively, this system can streamline the acquisition of multimodal piano performance datasets.
