arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.03307cs.SD
收藏

GRAM:用于现实环境的空间通用音频表示

GRAM: Spatial general-purpose audio representations for real-world environments

Goksenin Yuksel, Marcel van Gerven, Kiki van der Heijden

AI总结:

GRAM通过多通道掩码自编码器学习空间音频表示,在真实环境中的声音定位和识别任务中表现出色。

摘要:

音频基础模型学习通用的音频表示,以促进广泛的任务。尽管这些模型在传统单通道、干燥音频片段上的性能大幅提高,但在具有混响和噪声的真实环境中的表现有限。此外,大多数音频基础模型忽略了真实环境中的空间维度,排除了涉及声音定位的任务。为了解决这些限制,我们提出了GRAM:一个通用的真实环境音频模型,采用多通道掩码自编码器高效学习空间音频表示。我们评估了GRAM和其他音频基础模型在标准化的高保真自然空间声学环境模拟以及真实环境录音上的表现,并发布了这两个互补的基准任务集:NatHEAR和RealSELD。我们的结果表明,GRAM在NatHEAR和干净的单通道版本HEAR上均优于最先进的自监督音频基础模型,同时仅使用少量训练数据。GRAM在模拟环境中也展示了最先进的定位性能,并能高效地推广到RealSELD中的真实环境录音。综合来看,GRAM为现实环境中的稳健空间音频基础模型迈出了重要一步。

英文摘要:

Audio foundation models learn general-purpose audio representations that facilitate a wide range of downstream tasks. While the performance of these models has greatly increased for conventional single-channel, dry audio clips, their success in real-world acoustic environments with reverberation and noise is limited. Furthermore, most audio foundation models ignore the spatial dimension of real-world acoustic environments, ruling out tasks involving sound localization. To address these limitations, we propose GRAM: a general-purpose real-world audio model that employs a multi-channel masked autoencoder to efficiently learn spatial audio representations. We evaluated GRAM and other audio foundation models in a standardized manner on high-quality simulations of naturalistic, spatial acoustic environments as well as recordings of real-world environments and release these two complementary benchmark task suites: NatHEAR and RealSELD. Our results demonstrate that GRAM outperforms all state-of-the-art self-supervised audio foundation models on NatHEAR and the clean, single-channel version HEAR, while using only a fraction of the training data. GRAM also shows state-of-the-art localization performance in simulated environments and generalizes efficiently to real-world recordings in RealSELD. Taken together, GRAM presents a significant advance toward robust spatial audio foundation models for real-world environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑