SyncViolinist：基于运弓与按指的面向音乐的小提琴动作生成
SyncViolinist: Music-Oriented Violin Motion Generation Based on Bowing and Fingering
AI总结：
针对现有音频驱动小提琴动作生成需额外输入、难以兼顾全局与细粒度特征的问题，提出多阶段端到端框架SyncViolinist，仅用音频即可生成同步演奏动作，在主客观实验中均优于现有最优方法。
摘要：
自动生成逼真的音乐演奏动作可极大提升数字媒体制作效率，这类工作通常需要专业人士与音乐家协作完成。然而，捕捉精准音乐演奏所需的复杂身体、手部与手指动作颇具挑战。现有方法常因音频与动作间的复杂映射关系存在不足，通常需要乐谱或MIDI数据等额外输入。\n本文提出SyncViolinist，这是一个多阶段端到端框架，仅需音频输入即可生成同步的小提琴演奏动作。我们的方法通过两个核心模块——运弓/按指模块与动作生成模块，解决了同时捕捉全局与细粒度演奏特征的难题。运弓/按指模块从音频中提取详细的演奏信息，动作生成模块利用这些信息生成精准协调的身体动作，契合小提琴演奏的时间粒度与特性。\n我们在未见过的小提琴演奏音频上开展实验，SyncViolinist的定性与定量结果均显著提升，性能优于现有最优方法。由专业小提琴手参与的大量主观评估进一步验证了我们方法的有效性。代码与数据集可在https://github.com/Kakanat/SyncViolinist获取。
英文摘要：
Automatically generating realistic musical performance motion can greatly enhance digital media production, often involving collaboration between professionals and musicians. However, capturing the intricate body, hand, and finger movements required for accurate musical performances is challenging. Existing methods often fall short due to the complex mapping between audio and motion, typically requiring additional inputs like scores or MIDI data. In this work, we present SyncViolinist, a multi-stage end-to-end framework that generates synchronized violin performance motion solely from audio input. Our method overcomes the challenge of capturing both global and fine-grained performance features through two key modules: a bowing/fingering module and a motion generation module. The bowing/fingering module extracts detailed playing information from the audio, which the motion generation module uses to create precise, coordinated body motions reflecting the temporal granularity and nature of the violin performance. We demonstrate the effectiveness of SyncViolinist with significantly improved qualitative and quantitative results from unseen violin performance audio, outperforming state-of-the-art methods. Extensive subjective evaluations involving professional violinists further validate our approach. The code and dataset are available at https://github.com/Kakanat/SyncViolinist.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
