arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.00282eess.AScs.SD
收藏

PAM: 提示音频-语言模型用于音频质量评估

PAM: Prompting Audio-Language Models for Audio Quality Assessment

Carnegie Mellon University(卡内基梅隆大学);Microsoft(微软公司)

Soham Deshmukh, Dareen Alharthi, Benjamin Elizalde, Hannes Gamper, Mahmoud Al Ismail, Rita Singh, Bhiksha Raj, Huaming Wang

AI总结:

本研究提出基于音频-语言模型的无参考音频质量评估指标PAM,无需参考数据集嵌入或任务特定模型训练,在四类音频任务上验证了其与现有指标及人类评分的良好相关性。

摘要:

尽管音频质量是包括生成建模在内的各类音频处理任务的关键性能指标,但其客观测量仍是一项挑战。音频-语言模型(Audio-Language Models, ALMs)在音频-文本对上预训练,这些配对数据可能包含音频质量、失真存在情况或噪声相关信息。给定音频输入和与质量相关的文本提示,ALM可用于计算两者之间的相似度分数。本文利用这一能力,提出了PAM——一种用于不同音频处理任务音频质量评估的无参考指标。与其他“无参考”指标不同,PAM既不需要在参考数据集上计算嵌入,也无需基于成本高昂的人类听力评分集训练任务特定模型。我们在文本转音频(text-to-audio, TTA)、文本转音乐生成(text-to-music generation, TTM)、文本转语音(text-to-speech, TTS)和深度噪声抑制(deep noise suppression, DNS)四项任务上,将PAM与现有成熟指标及人类听力评分进行对比,广泛评估了其可靠性。我们还通过受控失真、真实场景设置和提示选择开展了多项消融研究。评估结果表明,PAM与现有指标及人类听力评分具有良好的相关性。这些结果证明了ALMs在构建通用音频质量指标方面的潜力。

英文摘要:

While audio quality is a key performance metric for various audio processing tasks, including generative modeling, its objective measurement remains a challenge. Audio-Language Models (ALMs) are pre-trained on audio-text pairs that may contain information about audio quality, the presence of artifacts, or noise. Given an audio input and a text prompt related to quality, an ALM can be used to calculate a similarity score between the two. Here, we exploit this capability and introduce PAM, a no-reference metric for assessing audio quality for different audio processing tasks. Contrary to other "reference-free" metrics, PAM does not require computing embeddings on a reference dataset nor training a task-specific model on a costly set of human listening scores. We extensively evaluate the reliability of PAM against established metrics and human listening scores on four tasks: text-to-audio (TTA), text-to-music generation (TTM), text-to-speech (TTS), and deep noise suppression (DNS). We perform multiple ablation studies with controlled distortions, in-the-wild setups, and prompt choices. Our evaluation shows that PAM correlates well with existing metrics and human listening scores. These results demonstrate the potential of ALMs for computing a general-purpose audio quality metric.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑