arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.07530cs.SDcs.AIeess.AS
收藏

基于生成式基础模型的音频解释合成

Audio Explanation Synthesis with Generative Foundation Models

Imperial College London(帝国理工学院);Technical University of Munich(慕尼黑工业大学)

Alican Akman, Qiyang Sun, Björn W. Schuller

AI总结:

本文提出一种利用音频基础模型生成能力的新颖音频解释方法,通过整合特征归因技术识别嵌入空间中的重要特征并生成可听音频解释,在关键词检测和语音情感识别数据集上验证了其有效性。

摘要:

音频基础模型在各种任务中的成功日益增加,导致对改进可解释性以更好理解其复杂决策过程的需求不断增长。现有方法主要通过根据输入空间中元素对最终决策的影响来为其分配重要性,从而解释这些模型。在本文中,我们引入了一种新颖的音频解释方法,该方法利用了音频基础模型的生成能力。我们的方法通过整合已有的特征归因技术来识别该空间中的重要特征,从而利用这些模型内嵌入空间固有的表示能力。该方法随后通过优先处理最重要的特征来生成可听的音频解释。通过在包括关键词检测和语音情感识别在内的标准数据集上进行严格基准测试,我们的模型证明了其在生成音频解释方面的有效性。

英文摘要:

The increasing success of audio foundation models across various tasks has led to a growing need for improved interpretability to understand their intricate decision-making processes better. Existing methods primarily focus on explaining these models by attributing importance to elements within the input space based on their influence on the final decision. In this paper, we introduce a novel audio explanation method that capitalises on the generative capacity of audio foundation models. Our method leverages the intrinsic representational power of the embedding space within these models by integrating established feature attribution techniques to identify significant features in this space. The method then generates listenable audio explanations by prioritising the most important features. Through rigorous benchmarking against standard datasets, including keyword spotting and speech emotion recognition, our model demonstrates its efficacy in producing audio explanations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑