arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.15672cs.SDeess.AS
收藏

计算机听觉:从任务特定机器学习到基础模型

Computer Audition: From Task-Specific Machine Learning to Foundation Models

Technical University of Munich(慕尼黑工业大学);University of Augsburg(奥格斯堡大学);Imperial College(帝国理工学院)

Andreas Triantafyllopoulos, Iosif Tsangko, Alexander Gebhard, Annamaria Mesaros, Tuomas Virtanen, Björn Schuller

AI总结:

本文综述计算音频分析从传统任务特定流程向听觉基础模型的转变,阐明其核心运行原理及整合多任务、跨模态知识与人机交互的优势。

摘要:

基础模型(FMs)正日益引领计算机听觉(即利用机器理解声音)范畴内多种任务的最新进展。与传统流程相比,它们具有多项优势:其中包括将多个任务整合到单一模型中的能力、利用来自其他模态知识的选择,以及与人类用户随时可用的交互。自然地,这些前景在音频社区引发了巨大兴奋,并催生了一波构建新型通用音频基础模型的早期尝试。在本文中,我们概述了计算音频分析从传统流程向听觉基础模型过渡的过程。我们的工作强调了支撑这些模型的关键运行原理,并展示了它们如何容纳音频社区此前分别处理的多个任务。

英文摘要:

Foundation models (FMs) are increasingly spearheading recent advances on a variety of tasks that fall under the purview of computer audition -- the use of machines to understand sounds. They feature several advantages over traditional pipelines: among others, the ability to consolidate multiple tasks in a single model, the option to leverage knowledge from other modalities, and the readily-available interaction with human users. Naturally, these promises have created substantial excitement in the audio community, and have led to a wave of early attempts to build new, general-purpose foundation models for audio. In the present contribution, we give an overview of computational audio analysis as it transitions from traditional pipelines towards auditory foundation models. Our work highlights the key operating principles that underpin those models, and showcases how they can accommodate multiple tasks that the audio community previously tackled separately.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑