语音大语言模型中上下文与副语言推理的基准测试：基于野外数据的案例研究
Benchmarking Contextual and Paralinguistic Reasoning in Speech-LLMs: A Case Study with In-the-Wild Data
AI总结：
针对语音大语言模型在副语言推理上的不足，提出CP-Bench基准，通过两个QA数据集评估模型整合言语与非言语线索的能力，揭示现有评估空白，为构建情感智能语音模型提供方向。
摘要：
近期的语音大语言模型（speech-LLMs）在转录和翻译等任务中表现出色，但在理解对社交和情感智能至关重要的语音副语言方面仍存在局限。我们提出了CP-Bench，一个用于评估语音大语言模型在上下文副语言推理（即言语内容与情感和韵律等非言语线索的整合）能力的基准。该基准包含两个精心策划的问答（QA）数据集，要求同时具备语言理解和共情理解能力。我们评估了来自开源和闭源模型的最先进语音大语言模型，并针对不同问题类型进行了全面分析。排名前两位的模型进一步在温度调节下进行了分析，以了解其对该任务的影响。我们的基准揭示了现有评估中的一个关键空白，并为构建更具上下文感知和情感智能的语音大语言模型提供了见解。
英文摘要：
Recent speech-LLMs have shown impressive performance in tasks like transcription and translation, yet they remain limited in understanding the paralinguistic aspects of speech crucial for social and emotional intelligence. We propose CP-Bench, a benchmark for evaluating speech-LLMs on contextual paralinguistic reasoning the integration of verbal content with non-verbal cues like emotion and prosody. The benchmark includes two curated question answering (QA) datasets requiring both linguistic and empathetic understanding. We evaluate state-of-the-art speech-LLMs from both open and closed-source models and perform a comprehensive analysis across different question types. The top two models were further analyzed under temperature tuning to understand its effect on this task. Our benchmark reveals a key gap in existing evaluations and offers insights into building more context-aware and emotionally intelligent speech-capable LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
