Eureka-Audio: 在紧凑语言模型中触发音频智能
Eureka-Audio: Triggering Audio Intelligence in Compact Language Models
AI总结:
Eureka-Audio通过紧凑架构和统一端到端设计,在低参数量下实现高性能音频理解,匹配甚至超越大模型表现。
摘要:
我们提出了Eureka-Audio,一个紧凑且高性能的音频语言模型,其在广泛的音频理解基准测试中能够与4到18倍更大的模型竞争。尽管仅包含1.7B个参数,Eureka-Audio在自动语音识别(ASR)、音频理解和密集音频描述中表现出强劲的性能,匹配或超越了多个7B到30B的音频和多模态基线。该模型采用了一个统一的端到端架构,由轻量级语言主干、基于Whisper的音频编码器和一个稀疏激活的专家混合(MoE)适配器组成,该适配器显式考虑了音频异质性,并在有限容量下缓解了跨模态优化冲突。为了进一步增强言语特征推理,我们引入了DataFlux,一个闭环音频指令数据合成和验证管道,从原始音频中构建高质量且逻辑一致的监督。在ASR、知识推理、安全、指令遵循和言语特征基准上的广泛评估表明,Eureka-Audio在计算成本和性能之间实现了高效的平衡。这些结果使Eureka Audio成为轻量级音频理解模型的强大且实用的基线。
英文摘要:
We present Eureka-Audio, a compact yet high-performance audio language model that achieves competitive performance against models that are 4 to 18 times larger across a broad range of audio understanding benchmarks. Despite containing only 1.7B parameters, Eureka-Audio demonstrates strong performance on automatic speech recognition (ASR), audio understanding, and dense audio captioning, matching or surpassing multiple 7B to 30B audio and omni-modal baselines. The model adopts a unified end-to-end architecture composed of a lightweight language backbone, a Whisper-based audio encoder, and a sparsely activated Mixture-of-Experts (MoE) adapter that explicitly accounts for audio heterogeneity and alleviates cross-modal optimization conflicts under limited capacity. To further enhance paralinguistic reasoning, we introduce DataFlux, a closed loop audio instruction data synthesis and verification pipeline that constructs high quality, logically consistent supervision from raw audio. Extensive evaluations across ASR, knowledge reasoning, safety, instruction following, and paralinguistic benchmarks, demonstrate that Eureka-Audio achieves an efficient balance between computational cost and performance. These results establish Eureka Audio as a strong and practical baseline for lightweight audio understanding models.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。