CAF-Score: 通过LALMs校准CLAP用于无参考音频描述评估
CAF-Score: Calibrating CLAP with LALMs for Reference-free Audio Captioning Evaluation
AI总结：
本文提出CAF-Score，一种无参考音频描述评估指标，通过结合对比音频-文本嵌入与LALM推理，有效检测语法不一致和细微幻觉，实验表明其在BRACE基准上与人类判断相关性最高。
摘要：
尽管大型音频-语言模型（LALMs）在音频描述任务上有所进展，但稳健的评估仍具挑战性。基于参考的指标成本高且难以评估声音保真度，而基于对比语言-音频预训练（CLAP）的方法常忽视语法错误和细节。我们提出CAF-Score，一种无参考指标，通过校准CLAP的粗粒度语义对齐与LALMs的细粒度理解和语法意识。通过结合对比音频-文本嵌入与LALM推理，CAF-Score有效检测语法不一致和细微幻觉。在BRACE基准上的实验表明，我们的方法在与人类判断的相关性上最高，甚至在挑战性场景中优于基于参考的基线。这些结果凸显了CAF-Score在无参考音频描述评估中的有效性。代码和结果可在https://github.com/inseong00/CAF-Score获取。
英文摘要：
While Large Audio-Language Models (LALMs) have advanced audio captioning, robust evaluation remains difficult. Reference-based metrics are expensive and often fail to assess acoustic fidelity, while Contrastive Language-Audio Pretraining (CLAP)-based approaches frequently overlook syntactic errors and fine-grained details. We propose CAF-Score, a reference-free metric that calibrates CLAP's coarse-grained semantic alignment with the fine-grained comprehension and syntactic awareness of LALMs. By combining contrastive audio-text embeddings with LALM reasoning, CAF-Score effectively detects syntactic inconsistencies and subtle hallucinations. Experiments on the BRACE benchmark demonstrate that our approach achieves the highest correlation with human judgments, even outperforming reference-based baselines in challenging scenarios. These results highlight the efficacy of CAF-Score for reference-free audio captioning evaluation. Code and results are available at https://github.com/inseong00/CAF-Score.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
