利用知识蒸馏在语音情感识别中 leverage 未标记音视频数据
Leveraging Unlabeled Audio-Visual Data in Speech Emotion Recognition using Knowledge Distillation
AI总结：
针对语音情感识别标注数据成本高的问题，提出基于知识蒸馏的LiSER框架，借助大型教师模型利用未标记音视频数据将情绪知识迁移到轻量学生模型，降低对标注数据的依赖。
摘要：
作为人机交互系统核心组成的语音接口可借助语音情感识别（SER）根据用户情绪定制响应。由于人类通过多模态音视频线索传递情绪，开发融合两种模态的SER系统具有优势，但收集大量标注数据成本高昂。本文提出名为LightweightSER（LiSER，轻量级SER）的知识蒸馏框架，利用基于先进语音与人脸表征模型构建的大型教师模型，将未标记音视频数据用于SER。LiSER将语音情绪与面部表情相关知识从教师模型迁移到轻量级学生模型。在RAVDESS和CREMA-D两个基准数据集上的实验表明，LiSER可降低SER任务对大规模标注数据集的依赖。
英文摘要：
Voice interfaces integral to the human-computer interaction systems can benefit from speech emotion recognition (SER) to customize responses based on user emotions. Since humans convey emotions through multi-modal audio-visual cues, developing SER systems using both the modalities is beneficial. However, collecting a vast amount of labeled data for their development is expensive. This paper proposes a knowledge distillation framework called LightweightSER (LiSER) that leverages unlabeled audio-visual data for SER, using large teacher models built on advanced speech and face representation models. LiSER transfers knowledge regarding speech emotions and facial expressions from the teacher models to lightweight student models. Experiments conducted on two benchmark datasets, RAVDESS and CREMA-D, demonstrate that LiSER can reduce the dependence on extensive labeled datasets for SER tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
