SpeechIQ: 大型语言模型在语音理解中的跨认知层级语音智能商
SpeechIQ: Speech-Agentic Intelligence Quotient Across Cognitive Levels in Voice Understanding by Large Language Models
AI总结：
SpeechIQ是一种基于语音的智能评估框架，通过三个认知层级评估大型语言模型在语音理解中的能力，提供统一的比较和识别标注错误与幻觉。
摘要：
我们介绍了一种新的基于语音的智能商（SIQ）作为人类认知启发的评估流程，用于评估语音理解大型语言模型（LLM Voice）的能力。超越流行的语音理解度量标准，如词错误率（WER），SIQ通过布卢姆分类法的三个认知层级来评估LLM Voice：（1）记忆（即verbatim准确性的WER）；（2）理解（即LLM解释的一致性）；以及（3）应用（即模拟下游任务的问答准确性）。我们证明SIQ不仅量化了语音理解能力，还提供了统一的比较，比较级联方法（如ASR LLM）和端到端模型，识别现有基准中的标注错误，并检测LLM Voice中的幻觉。我们的框架代表了一种首次的智能评估，将认知原理与语音导向的基准相结合，同时揭示了多模态训练中被忽视的挑战。我们的代码和数据将开源以鼓励未来的研究。
英文摘要：
We introduce Speech-based Intelligence Quotient (SIQ) as a new form of human cognition-inspired evaluation pipeline for voice understanding large language models, LLM Voice, designed to assess their voice understanding ability. Moving beyond popular voice understanding metrics such as word error rate (WER), SIQ examines LLM Voice across three cognitive levels motivated by Bloom's Taxonomy: (1) Remembering (i.e., WER for verbatim accuracy); (2) Understanding (i.e., similarity of LLM's interpretations); and (3) Application (i.e., QA accuracy for simulating downstream tasks). We demonstrate that SIQ not only quantifies voice understanding abilities but also provides unified comparisons between cascaded methods (e.g., ASR LLM) and end-to-end models, identifies annotation errors in existing benchmarks, and detects hallucinations in LLM Voice. Our framework represents a first-of-its-kind intelligence examination that bridges cognitive principles with voice-oriented benchmarks, while exposing overlooked challenges in multi-modal training. Our code and data will be open source to encourage future studies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
