arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.12466cs.SDeess.AS
收藏

AudioEditor:一种基于扩散模型的免训练音频编辑框架

AudioEditor: A Training-Free Diffusion-Based Audio Editing Framework

College of Computer Science, Nankai University(南开大学计算机学院);Lingxi (Beijing) Technology Co., Ltd.(灵犀(北京)科技有限公司)

Yuhang Jia, Yang Chen, Jinghua Zhao, Shiwan Zhao, Wenjia Zeng, Yong Chen, Yong Qin

AI总结:

本文提出免训练音频编辑框架AudioEditor,通过Null-text Inversion和EOT-suppression方法在预训练扩散TTA模型上实现精确编辑并保留未编辑部分,实验验证其高质量编辑效果。

摘要:

基于扩散的文本到音频(TTA)生成已取得实质性进展,利用潜在扩散模型(LDM)生成高质量、多样化且与指令相关的音频。然而,除生成之外,音频编辑任务同样重要,但受到的关注相对较少。音频编辑任务面临两个主要挑战:执行精确编辑和保留未编辑部分。虽然基于LDM的工作流已在图像处理领域有效解决了这些挑战,但类似方法很少应用于音频编辑。本文提出了AudioEditor,一个基于预训练扩散式TTA模型构建的免训练音频编辑框架。AudioEditor融合了Null-text Inversion和EOT-suppression方法,使模型能够在执行准确编辑的同时保留原始音频特征。全面的客观和主观实验验证了AudioEditor在提供高质量音频编辑方面的有效性。代码和演示可在https://github.com/NKU-HLT/AudioEditor获取。

英文摘要:

Diffusion-based text-to-audio (TTA) generation has made substantial progress, leveraging latent diffusion model (LDM) to produce high-quality, diverse and instruction-relevant audios. However, beyond generation, the task of audio editing remains equally important but has received comparatively little attention. Audio editing tasks face two primary challenges: executing precise edits and preserving the unedited sections. While workflows based on LDMs have effectively addressed these challenges in the field of image processing, similar approaches have been scarcely applied to audio editing. In this paper, we introduce AudioEditor, a training-free audio editing framework built on the pretrained diffusion-based TTA model. AudioEditor incorporates Null-text Inversion and EOT-suppression methods, enabling the model to preserve original audio features while executing accurate edits. Comprehensive objective and subjective experiments validate the effectiveness of AudioEditor in delivering high-quality audio edits. Code and demo can be found at https://github.com/NKU-HLT/AudioEditor.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑