Perception of auditory events is inherently multimodal relying on both audio and visual cues. A large number of existing multimodal approaches process each modality using modality-specific models and then fuse the embeddings to encode the joint information. In contrast, we employ heterogeneous graphs to explicitly capture the spatial and temporal relationships between the modalities and represent detailed information about the underlying signal. Using heterogeneous graph approaches to address the task of visually-aware acoustic event classification, which serves as a compact, efficient and scalable way to represent data in the form of graphs. Through heterogeneous graphs, we show efficiently modelling of intra- and inter-modality relationships both at spatial and temporal scales. Our model can easily be adapted to different scales of events through relevant hyperparameters. Experiments on AudioSet, a large benchmark, shows that our model achieves state-of-the-art performance.
MTDA-HSED: Mutual-Assistance Tuning and Dual-Branch Aggregating for Heterogeneous Sound Event Detection
School of Artificial Intelligence, Beijing University of Posts and Telecommunications(北京邮电大学人工智能学院);School of Intelligent Engineering and Automation, Beijing University of Posts and Telecommunications(北京邮电大学智能工程与自动化学院)
The Unreliability of Acoustic Systems in Alzheimer's Speech Datasets with Heterogeneous Recording Conditions
Instituto de Investigación en Ciencias de la Computación, UBA-CONICET(布宜诺斯艾利斯大学-国家科学技术研究委员会计算机科学研究所);Departamento de Computación, Facultad de Ciencias Exactas y Naturales, UBA(布宜诺斯艾利斯大学精确与自然科学学院计算机系);Centro de Neurociencias Cognitivas, Universidad de San Andrés(圣安德烈斯大学认知神经科学中心)
End-to-End Speech Recognition from Federated Acoustic Models
University of Cambridge(剑桥大学);Avignon University(阿维尼翁大学);Telecom Paris(巴黎电信);University of Oxford(牛津大学);Adap GmbH(Adap有限责任公司)
在联邦学习(FL)环境下训练自动语音识别(ASR)模型近年来引起了广泛的关注。然而,在文献中的FL场景往往是人为的,并未能捕捉到真实FL系统的复杂性。在本文中,我们构建了一个具有挑战性和现实的ASR联邦实验设置,包括客户端的异构数据分布使用法语和意大利语集的CommonVoice数据集,一个大型的异构数据集,包含数千个不同的扬声器,声学环境和噪声。我们提出了第一个基于注意力的序列到序列端到端(E2 E)ASR模型与三个聚合加权策略-标准FedAvg,基于损失的聚合和一个新的字出错率(WER)为基础的聚合,在两个现实的FL场景:跨筒仓与10个客户端和跨设备与2K和4K客户端进行比较的实证研究。我们的分析E2 E ASR异构和现实的联邦声学模型提供了基础,为未来的研究和开发现实的FL为基础的ASR应用。
Acoustic UAV Detection in Battlefield Scenarios: Handling Noise, Domain Shift, and Weak Labels
Institute for Applied System Analysis, Igor Sikorsky Kyiv Polytechnic Institute(伊戈尔·西科尔斯基基辅理工学院应用系统分析研究所);Zvook(兹沃克公司);Kyiv School of Economics(基辅经济学院)
From Signals to Evidence: Interpretable Underwater Acoustic Target Sensing with Large Language Models - Part I: Framework and Recognition
School of Artificial Intelligence, Northwestern Polytechnical University(西北工业大学人工智能学院);School of Ocean Engineering and Technology, Sun Yat-sen University(中山大学海洋工程与技术学院);School of Marine Science and Technology, Northwestern Polytechnical University(西北工业大学海洋科学与技术学院);School of Electronic and Optical Engineering, Nanjing University of Science and Technology(南京理工大学电子与光学工程学院)