AI 中文总结
该研究提出书法驱动的实时生成式音乐表演界面Calliphony，构建低延迟流水线捕捉毛笔运动映射为控制信号，实现多轨MIDI生成，将书法扩展为视听AI辅助表演场景，为现场音乐表演提供新方式。
AI 中文摘要
尽管音乐生成模型近年来受到了广泛关注，但如何将其有效融入现场音乐表演仍需进一步探索。本文提出了Calliphony，一种用于实时生成式音乐表演的书法驱动界面。具体而言，我们构建了一个低延迟流水线，通过可附加传感器捕捉毛笔运动，并将其映射为控制信号以实现实时符号音乐生成。在表演场景中，系统利用生成模型生成多轨MIDI，同时由毛笔衍生的控制信号约束事件时序并激活额外音乐层。生成的旋律随后通过实时和声与额外声部扩展，最终通过DAW进行现场编排。Calliphony的贡献包括：（1）一个面向表演的原型，将书法运动作为外部控制层用于实时符号音乐生成模型，控制音符密度、音高约束及伴奏层激活；（2）一个跨模态表演场景，将书法从主要视觉实践扩展为视听、AI辅助的场景。
英文摘要
While music generative models have recently gained significant attention, how they can be effectively integrated into live music performances still requires further exploration. This paper presents Calliphony, a calligraphy-driven interface for real-time generative music performance. Specifically, we build a low-latency pipeline that captures brush motion with an attachable sensor and maps it to control signals for real-time symbolic music generation. Using a generative model, the system produces multi-track MIDI in performance settings, while brush-derived control signals constrain event timing and activate additional musical layers. The generated melody is then extended with real-time harmony and additional voices, and finally rendered through a DAW for live staging. Calliphony contributes: (1) a performance-oriented prototype that uses calligraphic motion as an external control layer for a real-time symbolic music generation model, controlling note density, pitch constraints, and accompaniment-layer activation; and (2) a cross-modal performance scenario that extends calligraphy beyond a primarily visual practice into an audiovisual, AI-assisted setting.
Comments6 pages, 4 figures. Published in the Proceedings of the International Conference on New Interfaces for Musical Expression (NIME 2026). The first two authors contributed equally
Journal refProceedings of the International Conference on New Interfaces for Musical Expression (NIME 2026), 2026
