SAKURA：基于语音和音频信息的大型音频语言模型多跳推理研究
SAKURA: On the Multi-hop Reasoning of Large Audio-Language Models Based on Speech and Audio Information
AI总结：
针对大型音频语言模型（LALMs）多跳推理能力缺乏系统评估的问题，本文提出SAKURA基准，实验表明LALMs难以整合语音/音频信息进行多跳推理，为该领域研究提供了资源与方向。
摘要：
大型音频语言模型（LALMs）将大型语言模型扩展至语音、音频等多模态理解领域。尽管其在语音和音频处理任务上的表现已被广泛研究，但推理能力仍未得到充分探索，尤其是回忆和整合多个事实的多跳推理能力缺乏系统评估。现有基准聚焦于通用语音和音频处理任务、对话能力及公平性，却忽略了这一方面。为填补空白，我们引入SAKURA基准，用于评估LALMs基于语音和音频信息的多跳推理能力。结果显示，即使LALMs能正确提取相关信息，也难以整合语音/音频表示进行多跳推理，凸显了多模态推理的根本挑战。研究发现揭示了LALMs的关键局限，为未来研究提供了见解和资源。
英文摘要：
Large audio-language models (LALMs) extend the large language models with multimodal understanding in speech, audio, etc. While their performances on speech and audio-processing tasks are extensively studied, their reasoning abilities remain underexplored. Particularly, their multi-hop reasoning, the ability to recall and integrate multiple facts, lacks systematic evaluation. Existing benchmarks focus on general speech and audio-processing tasks, conversational abilities, and fairness but overlook this aspect. To bridge this gap, we introduce SAKURA, a benchmark assessing LALMs' multi-hop reasoning based on speech and audio information. Results show that LALMs struggle to integrate speech/audio representations for multi-hop reasoning, even when they extract the relevant information correctly, highlighting a fundamental challenge in multimodal reasoning. Our findings expose a critical limitation in LALMs, offering insights and resources for future research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
