arXiv 2505.139782025-11-27cs.SDeess.AS

弥合语音情感识别与性格：数据集和时间交互条件网络

Bridging Speech Emotion Recognition and Personality: Dataset and Temporal Interaction Condition Network

Department of Intelligence Science and Technology, School of Informatics, Kyoto University（智能科学与技术系，信息学院，京都大学）

Yuan Gao, Hao Shi, Yahui Fu, Chenhui Chu, Tatsuya Kawahara

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出PA-IEMOCAP数据集和TICN网络，通过整合人格特征提升语音情感识别的准确性，实验显示在无用户人格信息时，自动识别人格特征可提高情感价值识别的CCC达11.17%。

摘要：

本研究探讨了性格特质与情感表达之间的相互作用，探索了如何利用性格信息提高语音情感识别（SER）。我们收集了IEMOCAP数据集的人格注释，使其成为首个包含情感和人格注释（PA-IEMOCAP）的语音数据集，从而能够直接将性格特质整合到SER中。对这一数据集的统计分析确定了性格特质与情感表达之间存在显著相关性。为了提取细粒度的人格特征，我们提出了一个时间交互条件网络（TICN），其中人格特征与基于HuBERT的声学特征相结合用于SER。实验表明，结合真实的人格特征显著提高了情感价值的识别，与不包含人格信息的基线相比，一致性相关系数（CCC）从0.698提高到0.785。对于在对话系统中无法获取用户人格信息的实用应用，我们开发了一个前端模块，用于自动识别人格。使用这些自动预测的特征作为我们提出TICN模型的输入，我们实现了情感价值识别的CCC为0.776，比基线提高了11.17%。这些发现证实了具有意识的人格SER的有效性，并为进一步探索具有意识的人格语音处理应用提供了坚实的基础。

英文摘要：

This study investigates the interaction between personality traits and emotion expression, exploring how personality information can improve speech emotion recognition (SER). We collect the personality annotation for the IEMOCAP dataset, making it the first speech dataset that contains both emotion and personality annotations (PA-IEMOCAP), and enabling direct integration of personality traits into SER. Statistical analysis on this dataset identified significant correlations between personality traits and emotional expressions. To extract finegrained personality features, we propose a temporal interaction condition network (TICN), in which personality features are integrated with HuBERT-based acoustic features for SER. Experiments show that incorporating ground-truth personality traits significantly enhances valence recognition, improving the concordance correlation coefficient (CCC) from 0.698 to 0.785 compared to the baseline without personality information. For practical applications in dialogue systems where personality information about the user is unavailable, we develop a front-end module of automatic personality recognition. Using these automatically predicted traits as inputs to our proposed TICN model, we achieve a CCC of 0.776 for valence recognition, representing an 11.17% relative improvement over the baseline. These findings confirm the effectiveness of personality-aware SER and provide a solid foundation for further exploration in personality-aware speech processing applications.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。