arXiv 2511.049142025-11-12cs.SDcs.AI

MERaLiON-SER：适用于英语和东南亚语言的鲁棒语音情感识别模型

MERaLiON-SER: Robust Speech Emotion Recognition Model for English and SEA Languages

MERaLiON Team Institute for Infocomm Research (I 2 R), A*STAR, Singapore（MERaLiON团队信息与通信研究所（I 2 R），A*STAR，新加坡）

Hardik B. Sailor, Aw Ai Ti, Chen Fang Yih Nancy, Chiu Ying Lay, Ding Yang, He Yingxu, Jiang Ridong, Li Jingtao, Liao Jingyi, Liu Zhuohan, Lu Yanfeng, Ma Yi, Man… 展开作者

Hardik B. Sailor, Aw Ai Ti, Chen Fang Yih Nancy, Chiu Ying Lay, Ding Yang, He Yingxu, Jiang Ridong, Li Jingtao, Liao Jingyi, Liu Zhuohan, Lu Yanfeng, Ma Yi, Manas Gupta, Muhammad Huzaifah Bin Md Shahrin, Nabilah Binte Md Johan, Nattadaporn Lertcheva, Pan Chunlei, Pham Minh Duc, Siti Maryam Binte Ahmad Subaidi, Siti Umairah Binte Mohammad Salleh, Sun Shuo, Tarun Kumar Vangani, Wang Qiongqiong, Won Cheng Yi Lewis, Wong Heng Meng Jeremy, Wu Jinyang, Zhang Huayun, Zhang Longyin, Zou Xunlong

备注：: https://huggingface.co/MERaLiON/MERaLiON-SER-v1

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对英语和东南亚语言的语音情感识别问题，提出MERaLiON-SER模型，通过加权分类交叉熵与CCC损失的混合目标联合建模离散和维度情感，在多语言基准上优于现有模型，为情感感知智能体系统提供基础。

摘要：

我们提出了MERaLiON-SER，这是一个为英语和东南亚语言设计的鲁棒语音情感识别模型。该模型采用混合目标函数训练，结合加权分类交叉熵和一致性相关系数（CCC）损失，用于联合离散和维度情感建模。这种双重方法使模型既能捕捉不同的情感类别（如快乐或愤怒），又能捕捉细粒度的情感维度，如唤醒度（强度）、效价（积极性/消极性）和优势度（控制感），从而更全面、鲁棒地表示人类情感。通过对多语言新加坡语（英语、中文、马来语和泰米尔语）和其他公共基准的广泛评估，MERaLiON-SER持续优于开源语音编码器和大型Audio-LLM。这些结果强调了专门的纯语音模型在准确的副语言理解和跨语言泛化中的重要性。此外，该框架为将情感感知整合到未来的智能体音频系统中提供了基础，实现更具同理心和上下文适应性的多模态推理。

英文摘要：

We present MERaLiON-SER, a robust speech emotion recognition model designed for English and Southeast Asian languages. The model is trained using a hybrid objective combining weighted categorical cross-entropy and Concordance Correlation Coefficient (CCC) losses for joint discrete and dimensional emotion modelling. This dual approach enables the model to capture both the distinct categories of emotion (like happy or angry) and the fine-grained, such as arousal (intensity), valence (positivity/negativity), and dominance (sense of control), leading to a more comprehensive and robust representation of human affect. Extensive evaluations across multilingual Singaporean languages (English, Chinese, Malay, and Tamil ) and other public benchmarks show that MERaLiON-SER consistently surpasses both open-source speech encoders and large Audio-LLMs. These results underscore the importance of specialised speech-only models for accurate paralinguistic understanding and cross-lingual generalisation. Furthermore, the proposed framework provides a foundation for integrating emotion-aware perception into future agentic audio systems, enabling more empathetic and contextually adaptive multimodal reasoning.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。