arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.07260eess.AS
收藏

MorphFader:利用文本转音频模型实现细粒度可控变声

MorphFader: Enabling Fine-grained Controllable Morphing with Text-to-Audio Models

Purnima Kamath, Chitralekha Gupta, Suranga Nanayakkara

AI总结:

针对文本转音频模型难以细粒度控制声音语义以实现变声的问题,提出MorphFader方法,通过插值扩散过程中交叉注意力层组件实现不同提示声音的平滑变声,经客观指标和听测验证了其细粒度语义控制与平滑变声能力。

摘要:

声音变声是将一种声音逐渐平滑地转换为另一种声音,以生成同时兼具两者特征的新型感知混合声音的过程。近年来,基于扩散的文本转音频模型已能通过文本提示生成高质量声音。然而,变声所需的对声音语义的细粒度控制,使用文本实现颇具挑战。本文提出MorphFader,这是一种利用文本转音频模型对不同提示生成的声音进行变声的可控方法。通过在扩散过程中拦截并插值交叉注意力层的组件,我们能在不同文本提示生成的声音之间创建平滑的变声效果。借助客观指标和感知听力测试,我们证明了该方法能够细粒度控制声音语义并生成平滑的变声效果。

英文摘要:

Sound morphing is the process of gradually and smoothly transforming one sound into another to generate novel and perceptually hybrid sounds that simultaneously resemble both. Recently, diffusion-based text-to-audio models have produced high-quality sounds using text prompts. However, granularly controlling the semantics of the sound, which is necessary for morphing, can be challenging using text. In this paper, we propose \textit{MorphFader}, a controllable method for morphing sounds generated by disparate prompts using text-to-audio models. By intercepting and interpolating the components of the cross-attention layers within the diffusion process, we can create smooth morphs between sounds generated by different text prompts. Using both objective metrics and perceptual listening tests, we demonstrate the ability of our method to granularly control the semantics in the sound and generate smooth morphs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑