首页/论文/2003.00351

arXiv 2003.003512020-03cs.CVcs.LGcs.SDeess.AS

Emotion Recognition System from Speech and Visual Information based on Convolutional Neural Networks

University Politehnica of Bucharest（布加勒斯特理工大学）；Fotonation Romania, member of Xperi Group（Fotonation罗马尼亚公司（Xperi集团成员））

Nicolae-Catalin Ristea, Liviu Cristian Dutu, Anamaria Radoi

DOI：: 10.1109/SPED.2019.8906538

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

Emotion recognition has become an important field of research in the human-computer interactions domain. The latest advancements in the field show that combining visual with audio information lead to better results if compared to the case of using a single source of information separately. From a visual point of view, a human emotion can be recognized by analyzing the facial expression of the person. More precisely, the human emotion can be described through a combination of several Facial Action Units. In this paper, we propose a system that is able to recognize emotions with a high accuracy rate and in real time, based on deep Convolutional Neural Networks. In order to increase the accuracy of the recognition system, we analyze also the speech data and fuse the information coming from both sources, i.e., visual and audio. Experimental results show the effectiveness of the proposed scheme for emotion recognition and the importance of combining visual with audio data.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Interpretable Multimodal Emotion Recognition using Hybrid Fusion of Speech and Image Data

Indian Institute of Technology Roorkee（印度理工学院鲁尔基分校）

This paper proposes a multimodal emotion recognition system based on hybrid fusion that classifies the emotions depicted by speech utterances and corresponding images into discrete classes. A new interpretability technique has been developed to identify the important speech & image features leading to the prediction of particular emotion classes. The proposed system's architecture has been determined through intensive ablation studies. It fuses the speech & image features and then combines speech, image, and intermediate fusion outputs. The proposed interpretability technique incorporates the …

2023-01-07cs.CVcs.SDeess.AS

EmoTalk：面向3D人脸动画的语音驱动情感解耦

EmoTalk: Speech-Driven Emotional Disentanglement for 3D Face Animation

Renmin University of China（中国人民大学）；Nanjing University of Science and Technology（南京理工大学）；The Hong Kong University of Science and Technology（香港科技大学）；Psyche AI Inc.（赛琪人工智能公司）；Chinese Academy of Sciences（中国科学院）；Tsinghua University（清华大学）

针对语音驱动3D人脸动画中情感与内容难分离、情感表情不足的问题，提出含EDE的端到端网络，结合blendshapes监督与新数据集3D-ETF，生成可控且多样的情感3D人脸动画，效果优于现有最优方法。

2023-08-25cs.CVcs.SDeess.AS

Speech Driven Talking Face Generation from a Single Image and an Emotion Condition

University of Rochester（罗切斯特大学）

Visual emotion expression plays an important role in audiovisual speech communication. In this work, we propose a novel approach to rendering visual emotion expression in speech-driven talking face generation. Specifically, we design an end-to-end talking face generation system that takes a speech utterance, a single face image, and a categorical emotion label as input to render a talking face video synchronized with the speech and expressing the conditioned emotion. Objective evaluation on image quality, audiovisual synchronization, and visual emotion expression shows that the proposed system…

2021-07-21cs.CVcs.LGcs.MM

SEDTalker: 基于帧级语音情感分离的3D面部动画

SEDTalker: Emotion-Aware 3D Facial Animation Using Frame-Level Speech Emotion Diarization

University of Alberta, Edmonton, AB, Canada（阿尔伯塔大学）；University of Florence, Florence, Italy（佛罗伦萨大学）

SEDTalker通过帧级语音情感分离实现情感感知的3D面部动画，利用混合Transformer-Mamba架构生成连续表情控制。在多语种语音情感分离数据集和EmoVOCA数据集上验证，取得高精度情感识别和低重建误差。

2026-04-14cs.CV

FSER: Deep Convolutional Neural Networks for Speech Emotion Recognition

Jacobs University, Germany（雅各布斯大学，德国）；Drexel University, USA（德雷塞尔大学，美国）

Using mel-spectrograms over conventional MFCCs features, we assess the abilities of convolutional neural networks to accurately recognize and classify emotions from speech data. We introduce FSER, a speech emotion recognition model trained on four valid speech databases, achieving a high-classification accuracy of 95,05\%, over 8 different emotion classes: anger, anxiety, calm, disgust, happiness, neutral, sadness, surprise. On each benchmark dataset, FSER outperforms the best models introduced so far, achieving a state-of-the-art performance. We show that FSER stays reliable, independently of…

2021-09-15cs.CVcs.HCeess.AS

Emotion Recognition in Speech using Cross-Modal Transfer in the Wild

University of Oxford（牛津大学）

Obtaining large, human labelled speech datasets to train models for emotion recognition is a notoriously challenging task, hindered by annotation cost and label ambiguity. In this work, we consider the task of learning embeddings for speech classification without access to any form of labelled audio. We base our approach on a simple hypothesis: that the emotional content of speech correlates with the facial expression of the speaker. By exploiting this relationship, we show that annotations of expression can be transferred from the visual domain (faces) to the speech domain (voices) through cr…

2018-08-16cs.CV

更多相关论文收起更多论文

基于解耦潜在扩散的情感语音驱动3D人体动画

Emotional Speech-driven 3D Body Animation via Disentangled Latent Diffusion

KTH Royal Institute of Technology, Sweden（瑞典皇家理工学院）；Max Planck Institute for Intelligent Systems, Germany（德国马克斯·普朗克智能系统研究所）

本文提出AMUSE模型，通过解耦潜在向量实现情感语音驱动的3D人体动画生成，能控制情绪和风格，生成更符合语音内容且表现更真实的情绪动作序列。

2026-03-30cs.CV

Cross Lingual Speech Emotion Recognition: Urdu vs. Western Languages

Information Technology University (ITU)-Punjab（旁遮普信息技术大学）；COMSATS University Islamabad (CUI)（伊斯兰堡COMSATS大学）

Cross-lingual speech emotion recognition is an important task for practical applications. The performance of automatic speech emotion recognition systems degrades in cross-corpus scenarios, particularly in scenarios involving multiple languages or a previously unseen language such as Urdu for which limited or no data is available. In this study, we investigate the problem of cross-lingual emotion recognition for Urdu language and contribute URDU---the first ever spontaneous Urdu-language speech emotion database. Evaluations are performed using three different Western languages against Urdu and…

2020-07-28cs.CLcs.CV

基于内容-情绪解耦的情绪语音驱动动画

Emotional Speech-Driven Animation with Content-Emotion Disentanglement

Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；KTH（瑞典皇家理工学院）

本文提出EMOTE框架，通过语音与情绪解耦监督、内容-情绪交换和时序VAE动作先验，从语音生成口型同步且情绪可控的高质量3D说话头像动画。

2023-09-26cs.CVcs.GR