While audio quality is a key performance metric for various audio processing tasks, including generative modeling, its objective measurement remains a challenge. Audio-Language Models (ALMs) are pre-trained on audio-text pairs that may contain information about audio quality, the presence of artifacts, or noise. Given an audio input and a text prompt related to quality, an ALM can be used to calculate a similarity score between the two. Here, we exploit this capability and introduce PAM, a no-reference metric for assessing audio quality for different audio processing tasks. Contrary to other "reference-free" metrics, PAM does not require computing embeddings on a reference dataset nor training a task-specific model on a costly set of human listening scores. We extensively evaluate the reliability of PAM against established metrics and human listening scores on four tasks: text-to-audio (TTA), text-to-music generation (TTM), text-to-speech (TTS), and deep noise suppression (DNS). We perform multiple ablation studies with controlled distortions, in-the-wild setups, and prompt choices. Our evaluation shows that PAM correlates well with existing metrics and human listening scores. These results demonstrate the potential of ALMs for computing a general-purpose audio quality metric.
Enhancing Speech Large Language Models with Prompt-Aware Mixture of Audio Encoders
School of Computer Science and Engineering, Northeastern University(东北大学计算机科学与工程学院);Huawei Translation Services Center(华为翻译服务中心);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));College of Computer Science and Technology, Harbin Engineering University(哈尔滨工程大学计算机科学与技术学院);NiuTrans Research(NiuTrans研究院)
Beyond Accuracy: ARIA-Rubrics for Evaluating Audio Reasoning in Large Audio Language Models
Imperial College London(帝国理工学院);Technische Universität München(慕尼黑工业大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Shanghai Jiao Tong University(上海交通大学);Johns Hopkins University(约翰斯·霍普金斯大学)
Textual Acoustic Grounding for Generalizable LLM-Based Deepfake Voice Detection
German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);National Institute of Informatics(情报信息学研究所);Technical University of Berlin(柏林工业大学)
Selective Listening: Mechanism-Guided Control of Audio Influence in Large Audio-Language Models
College of Computer Science and Technology, National University of Defense Technology(国防科技大学计算机学院);State Key Laboratory of Complex & Critical Software Environment(复杂与关键软件环境国家重点实验室)
Unifying Speech Editing Detection and Content Localization via Prior-Enhanced Audio LLMs
Key Laboratory of Aerospace Information Security(航空信息安全与可信计算重点实验室);School of Cyber Science and Engineering(网络安全工程学院);Wuhan University(武汉大学);Independent Researcher(独立研究员);School of Computer Science and Technology(计算机科学与技术学院);Anhui University(安徽大学);Communication University of China(中国通信大学);Beihang University(北京航空航天大学)
Leveraging tropical reef, bird and unrelated sounds for superior transfer learning in marine bioacoustics
University College London(伦敦大学学院);Google DeepMind(谷歌DeepMind);Zoological Society of London(伦敦动物学会);Conservation Metrics Inc.(保护指标公司);University of New Hampshire(新罕布什尔大学);K. Lisa Yang Center for Conservation Bioacoustics, Cornell Lab of Ornithology, Cornell University(康奈尔大学鸟类学实验室K. Lisa Yang保护生物声学中心);Department of Public and Ecosystem Health, Cornell University(康奈尔大学公共与生态系统健康系);Sound Ocean Science(海洋声学科学);University of Exeter(埃克塞特大学);University of Bristol(布里斯托大学);IPB University(IPB大学);Google Research(谷歌研究院)
ALLM4ADD: Unlocking the Capabilities of Audio Large Language Models for Audio Deepfake Detection
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Department of Automation, Tsinghua University(清华大学自动化系);Taizhou University(台州大学);Anhui University(安徽大学);Beijing National Research Center for Information Science and Technology,Tsinghua University(北京信息科学与技术国家研究中心,清华大学)
A Survey of Audio Reasoning in Multimodal Foundation Models
The Chinese University of Hong Kong(香港中文大学);National Taiwan University(国立台湾大学);The University of Hong Kong(香港大学);The Hong Kong University of Science and Technology(香港科技大学)
Fine-Tuning Large Audio-Language Models with LoRA for Precise Temporal Localization of Prolonged Exposure Therapy Elements
1College of Information Sciences \& Technology, The Pennsylvania State University, USA 2Department of Psychiatry \& Behavioral Sciences, Emory University, USA 3School of Interactive Computing, Georgia Institute of Technology, USA 4School of Psychology, Georgia Institute of Technology, USA