OSUM-Pangu:基于OpenPangu在Ascend NPUs上的开源多维语音理解基础模型
OSUM-Pangu: An Open-Source Multidimension Speech Understanding Foundation Model Built upon OpenPangu on Ascend NPUs
AI总结:
OSUM-Pangu基于非CUDA的Ascend NPU平台,结合openPangu-7B LLM后端,实现语音理解任务的高准确率与自然语言交互能力。
摘要:
近年来,语音大语言模型的进展显著提升了多维语音理解能力。然而,大多数高性能框架主要针对GPU为中心的生态系统和专有后端进行优化,这在非CUDA计算基础设施上存在显著差距。本文提出了OSUM-Pangu,一个完全开源的语音理解基础模型,基于完全非CUDA的软件和硬件栈。通过将音频编码器与openPangu-7B LLM后端结合,我们成功在Ascend NPU平台上实现了完整的训练和推理流程。为在非CUDA资源限制下实现高效的任务对齐,我们采用了一种实用的训练过程,依次连接语音感知和用户意图识别。实验结果表明,OSUM-Pangu在任务准确率上与主流GPU模型相当,同时保持了稳健的自然语言交互能力。我们的工作为开源语音社区提供了一个可重复的、非CUDA的基准,推动了多模态智能的独立发展。
英文摘要:
Recent advancements in Speech Large Language Models have significantly enhanced multi-dimensional speech understanding. However, the majority of high-performance frameworks are predominantly optimized for GPU centric ecosystems and proprietary backbones, creating a significant gap for deployment on non-CUDA computing infrastructures. In this paper, we present OSUM-Pangu, a fully open-source speech understanding foundation model developed on a completely non-CUDA software and hardware stack. By integrating an audio encoder with the openPangu-7B LLM backbone, we successfully implement the entire training and inference pipeline on the Ascend NPU platform. To facilitate efficient task alignment under non-CUDA resource constraints, we adopt a practical training process that sequentially bridges speech perception and user intent recognition. Experimental results demonstrate that OSUM-Pangu achieves task accuracy comparable to mainstream GPU-based models while maintaining robust natural language interaction capabilities. Our work provides a reproducible, non-CUDA baseline for the open-source speech community, promoting the independent evolution of multimodal intelligence.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。