Salmon：声学语言模型评估套件
Salmon: A Suite for Acoustic Language Model Evaluation
AI总结：
针对现有语音语言模型评估缺乏多维度声学感知基准的问题，研究团队推出涵盖四类声学维度的SALMon评估套件，采用基于建模的高效评估方法，测试了多个模型的优劣势并公开相关资源。
摘要：
语音语言模型近来展现出作为通用语音处理系统的巨大潜力。这类模型能够对音频信号中存在的、除语音内容之外的丰富声学信息建模，例如情绪、背景噪声等。尽管如此，目前仍缺乏可评估模型对各类声学维度感知能力的评估基准。为填补这一空白，我们推出SALMon，这是一套全新的评估套件，涵盖背景噪声、情绪、说话人身份以及房间冲激响应四个维度。所提出的基准既评估被考察声学元素的一致性，也评估其与语音文本的匹配程度。我们采用基于建模的方法，衡量模型是否能给正确样本赋予比错误样本更高的分数。该方法使得即使针对大模型，基准的计算速度也很快。我们在SALMon上评估了多个语音语言模型，明确了每种被评估方法的优势与不足。我们已在https://pages.cs.huji.ac.il/adiyoss-lab/salmon/ 公开代码与数据。
英文摘要：
Speech language models have recently demonstrated great potential as universal speech processing systems. Such models have the ability to model the rich acoustic information existing in audio signals, beyond spoken content, such as emotion, background noise, etc. Despite this, evaluation benchmarks which evaluate awareness to a wide range of acoustic aspects, are lacking. To help bridge this gap, we introduce SALMon, a novel evaluation suite encompassing background noise, emotion, speaker identity and room impulse response. The proposed benchmarks both evaluate the consistency of the inspected element and how much it matches the spoken text. We follow a modelling based approach, measuring whether a model gives correct samples higher scores than incorrect ones. This approach makes the benchmark fast to compute even for large models. We evaluated several speech language models on SALMon, thus highlighting the strengths and weaknesses of each evaluated method. We make the code and data publicly available at https://pages.cs.huji.ac.il/adiyoss-lab/salmon/ .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
