arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.01929cs.SDcs.AIcs.LG
收藏

Woosh:一种声音效果基础模型

Woosh: A Sound Effects Foundation Model

Sony AI(索尼人工智能)

Gaëtan Hadjeres, Marc Ferras, Khaled Koutini, Benno Weck, Alexandre Bittar, Thomas Hummel, Zineb Lahrichi, Hakim Missoum, Joan Serrà, Yuki Mitsufuji

AI总结:

本文介绍了Sony AI发布的Woosh声音效果基础模型,提供高质量音频编码器/解码器、文本-音频对齐模型及文本到音频和视频到音频生成模型,展示了其在公开和私有数据上的竞争力。

摘要:

音频研究社区依赖于开放生成模型作为构建新方法和建立基准的基础工具。在本报告中,我们介绍了索尼AI公开发布的Woosh声音效果基础模型,详细介绍了其架构、训练过程以及与其他流行开放模型的评估。为了优化声音效果,我们提供了(1)高质量的音频编码器/解码器模型和(2)用于条件生成的文本-音频对齐模型,以及(3)文本到音频和(4)视频到音频生成模型。此外,还包含 distilled 的文本到音频和视频到音频模型,允许在低资源环境下运行和快速推理。我们在公开和私有数据上的评估显示,每个模块的性能在与现有开放替代品如StableAudio-Open和TangoFlux相比时具有竞争力或更好。推理代码和模型权重可在https://github.com/SonyResearch/Woosh上获得。演示样例可在https://sonyresearch.github.io/Woosh/上找到。

英文摘要:

The audio research community depends on open generative models as foundational tools for building novel approaches and establishing baselines. In this report, we present Woosh, Sony AI's publicly released sound effect foundation model, detailing its architecture, training process, and an evaluation against other popular open models. Being optimized for sound effects, we provide (1) a high-quality audio encoder/decoder model and (2) a text-audio alignment model for conditioning, together with (3) text-to-audio and (4) video-to-audio generative models. Distilled text-to-audio and video-to-audio models are also included in the release, allowing for low-resource operation and fast inference. Our evaluation on both public and private data shows competitive or better performance for each module when compared to existing open alternatives like StableAudio-Open and TangoFlux. Inference code and model weights are available at https://github.com/SonyResearch/Woosh. Demo samples can be found at https://sonyresearch.github.io/Woosh/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑