arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2311.09030eess.AScs.SD
收藏

基于AI的声景分析:联合识别声源与预测烦恼度

AI-based soundscape analysis: Jointly identifying sound sources and predicting annoyance

Ghent University(根特大学);Interuniversity Microelectronics Centre(校际微电子中心);University College London(伦敦大学学院)

Yuanbo Hou, Qiaoqiao Ren, Huizhong Zhang, Andrew Mitchell, Francesco Aletta, Jian Kang, Dick Botteldooren

AI总结:

本文提出基于交叉注意力融合的双分支卷积神经网络(DCNN-CaF),联合进行声源分类和烦恼度预测,在DeLTA数据集上优于传统方法,且与人类感知一致。

摘要:

声景研究通常试图通过调查用户来捕捉对声音环境的感知和理解。然而,对于长期监测或评估干预措施,需要基于声音信号的方法。为此,以往大多数研究集中于心理声学量或自动声音识别,很少有尝试纳入评价(例如在环形框架中)。本文提出了一种基于人工智能(AI)的双分支卷积神经网络与基于交叉注意力的融合(DCNN-CaF),用于分析自动声景表征,包括声音识别和评价。利用包含人工标注的声源标签和感知烦恼度的DeLTA数据集,提出了DCNN-CaF来执行声源分类(SSC)和人类感知烦恼度评级预测(ARP)。实验结果表明:(1)所提出的使用响度和Mel特征的DCNN-CaF优于仅使用其中一种特征的DCNN-CaF。(2)所提出的具有交叉注意力融合的DCNN-CaF在SSC和ARP任务上优于其他典型的基于AI的模型和与声景相关的传统机器学习方法。(3)相关性分析显示,人类与所提出的基于AI的DCNN-CaF模型在声源与烦恼度之间的关系上具有相似性。(4)泛化测试表明,在存在模型未知声源的情况下,所提出模型的ARP与专家预期一致,并能解释先前关于声景增强文献中的发现。

英文摘要:

Soundscape studies typically attempt to capture the perception and understanding of sonic environments by surveying users. However, for long-term monitoring or assessing interventions, sound-signal-based approaches are required. To this end, most previous research focused on psycho-acoustic quantities or automatic sound recognition. Few attempts were made to include appraisal (e.g., in circumplex frameworks). This paper proposes an artificial intelligence (AI)-based dual-branch convolutional neural network with cross-attention-based fusion (DCNN-CaF) to analyze automatic soundscape characterization, including sound recognition and appraisal. Using the DeLTA dataset containing human-annotated sound source labels and perceived annoyance, the DCNN-CaF is proposed to perform sound source classification (SSC) and human-perceived annoyance rating prediction (ARP). Experimental findings indicate that (1) the proposed DCNN-CaF using loudness and Mel features outperforms the DCNN-CaF using only one of them. (2) The proposed DCNN-CaF with cross-attention fusion outperforms other typical AI-based models and soundscape-related traditional machine learning methods on the SSC and ARP tasks. (3) Correlation analysis reveals that the relationship between sound sources and annoyance is similar for humans and the proposed AI-based DCNN-CaF model. (4) Generalization tests show that the proposed model's ARP in the presence of model-unknown sound sources is consistent with expert expectations and can explain previous findings from the literature on sound-scape augmentation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑