arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.02889cs.SDcs.CVcs.LGeess.AS
收藏

探索面向通用音频理解的联邦自监督学习

Exploring Federated Self-Supervised Learning for General Purpose Audio Understanding

TCL AI Lab(TCL人工智能实验室)

Yasar Abbas Ur Rehman, Kin Wai Lau, Yuyang Xie, Lan Ma, Jiajun Shen

AI总结:

本文提出联邦自监督学习框架FASSL,在非独立同分布的大规模异构客户端上学习音频特征,实现与集中式方法相当的检索性能,并助力最优全局模型聚合。

摘要:

联邦学习(FL)与自监督学习(SSL)的整合提供了一种独特且协同的组合,可在不损害用户数据隐私的前提下利用音频数据进行通用音频理解。然而,在联邦学习框架下针对通用音频理解研究自监督学习模型的工作甚少,尤其是当训练数据由大规模异构音频源生成时。本文评估了特征匹配和预测性音频自监督学习技术在基于非独立同分布(non-iid)数据模拟的大规模联邦学习设置中的性能。我们提出了一种新颖的联邦自监督学习(F-SSL)框架,称为FASSL,该框架能够从持有未标注音频数据的大规模去中心化异构客户端中学习中间特征表示。研究发现,音频联邦自监督学习方法在音频检索任务上的表现与集中式音频自监督学习方法相当。大量实验证明了FASSL的有效性和重要性,因为它有助于为最先进的联邦学习聚合方法获得最优全局模型。

英文摘要:

The integration of Federated Learning (FL) and Self-supervised Learning (SSL) offers a unique and synergetic combination to exploit the audio data for general-purpose audio understanding, without compromising user data privacy. However, rare efforts have been made to investigate the SSL models in the FL regime for general-purpose audio understanding, especially when the training data is generated by large-scale heterogeneous audio sources. In this paper, we evaluate the performance of feature-matching and predictive audio-SSL techniques when integrated into large-scale FL settings simulated with non-independently identically distributed (non-iid) data. We propose a novel Federated SSL (F-SSL) framework, dubbed FASSL, that enables learning intermediate feature representations from large-scale decentralized heterogeneous clients, holding unlabelled audio data. Our study has found that audio F-SSL approaches perform on par with the centralized audio-SSL approaches on the audio-retrieval task. Extensive experiments demonstrate the effectiveness and significance of FASSL as it assists in obtaining the optimal global model for state-of-the-art FL aggregation methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑