arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2312.04690cs.HCcs.AIcs.SDeess.AS
收藏

SynthScribe:用于合成器声音检索与探索的深度多模态工具

SynthScribe: Deep Multimodal Tools for Synthesizer Sound Retrieval and Exploration

University of Toronto(多伦多大学);Dalhousie University(达尔豪斯大学)

Stephen Brade, Bryan Wang, Mauricio Sousa, Gregory Lee Newsome, Sageev Oore, Tovi Grossman

AI总结:

SynthScribe 是一个基于多模态深度学习的全栈合成器声音系统,通过多模态搜索、用户偏好遗传算法和参数编辑支持,帮助音乐家高效检索、修改并创造全新声音。

摘要:

合成器是强大的工具,使音乐家能够创造动态且原创的声音。现有的合成器商业界面通常要求音乐家与复杂的低级参数交互,或管理庞大的预置声音库。为应对这些挑战,我们实现了 SynthScribe——一个全栈系统,利用多模态深度学习让用户在更高层次上表达意图。我们实现的功能解决了一系列困难,具体包括:1)在现有声音中进行搜索,2)创造全新的声音,3)对给定声音进行有意义的修改。这通过三个主要功能实现:一个面向大型合成器声音库的多模态搜索引擎;一个以用户为中心的遗传算法,可根据用户偏好创造并选择全新声音;一个声音编辑支持功能,针对基于文本或音频的查询,突出显示并给出关键控制参数的示例。用户研究结果表明,SynthScribe 能够可靠地检索和修改声音,同时提供创造全新声音的能力,从而拓展音乐家的创作视野。

英文摘要:

Synthesizers are powerful tools that allow musicians to create dynamic and original sounds. Existing commercial interfaces for synthesizers typically require musicians to interact with complex low-level parameters or to manage large libraries of premade sounds. To address these challenges, we implement SynthScribe -- a fullstack system that uses multimodal deep learning to let users express their intentions at a much higher level. We implement features which address a number of difficulties, namely 1) searching through existing sounds, 2) creating completely new sounds, 3) making meaningful modifications to a given sound. This is achieved with three main features: a multimodal search engine for a large library of synthesizer sounds; a user centered genetic algorithm by which completely new sounds can be created and selected given the users preferences; a sound editing support feature which highlights and gives examples for key control parameters with respect to a text or audio based query. The results of our user studies show SynthScribe is capable of reliably retrieving and modifying sounds while also affording the ability to create completely new sounds that expand a musicians creative horizon.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑