Time-continuous dimensional descriptions of emotions (e.g., arousal, valence) allow researchers to characterize short-time changes and to capture long-term trends in emotion expression. However, continuous emotion labels are generally not synchronized with the input speech signal due to delays caused by reaction-time, which is inherent in human evaluations. To deal with this challenge, we introduce a new convolutional neural network (multi-delay sinc network) that is able to simultaneously align and predict labels in an end-to-end manner. The proposed network is a stack of convolutional layers followed by an aligner network that aligns the speech signal and emotion labels. This network is implemented using a new convolutional layer that we introduce, the delayed sinc layer. It is a time-shifted low-pass (sinc) filter that uses a gradient-based algorithm to learn a single delay. Multiple delayed sinc layers can be used to compensate for a non-stationary delay that is a function of the acoustic space. We test the efficacy of this system on two common emotion datasets, RECOLA and SEWA, and show that this approach obtains state-of-the-art speech-only results by learning time-varying delays while predicting dimensional descriptors of emotions.
Video Soundtrack Generation by Aligning Emotions and Temporal Boundaries
Institute for Systems and Computer Engineering, Technology and Science (INESC TEC)(系统与计算机工程研究所(INESC TEC));ISEP, Polytechnic of Porto, School of Engineering(里斯本理工学院工程学院)
SEGAA: A Unified Approach to Predicting Age, Gender, and Emotion in Speech
School of Computer Science and Engineering(计算机科学与工程学院);Vellore Institute of Technology(韦洛尔理工学院);School of Electronics Engineering(电子工程学院);Voice Intelligence Group(语音智能集团);Samsung R&D Institute(三星研发研究院)
Your Face Mirrors Your Deepest Beliefs-Predicting Personality and Morals through Facial Emotion Recognition
MIT Center for Collective Intelligence(麻省理工学院集体智能中心);University of Perugia(佩鲁贾大学);Galaxyadvisors AG(Galaxyadvisors 公司);Lucerne University of Applied Sciences and Arts(卢塞恩应用科学与艺术大学);University of Cologne(科隆大学)
EmotionRankCLAP: Bridging Natural Language Speaking Styles and Ordinal Speech Emotion via Rank-N-Contrast
Center for Language and Speech Processing (CLSP), Johns Hopkins UniversityUSA(语言与语音处理中心(CLSP),约翰霍普金斯大学);The University of Texas at Dallas, USA(德克萨斯大学达拉斯分校);Amazon, USA(亚马逊);NUSSingapore(南洋理工大学);Language Technologies Institute (LTI), Carnegie Mellon UniversityUSA(语言技术研究所(LTI),卡内基梅隆大学)
Multi-view Multi-label Fine-grained Emotion Decoding from Human Brain Activity
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences(中国科学院脑科学与智能技术卓越创新中心)
Comparative Analysis of Mel-Frequency Cepstral Coefficients and Wavelet Based Audio Signal Processing for Emotion Detection and Mental Health Assessment in Spoken Speech
Predicting Therapeutic Outcome via Aligning Patient-Specific Knowledge Graph and Gene-Level Perturbation Representations
Interdisciplinary Program in Bioinformatics, Seoul National University(首尔国立大学生物信息学跨学科项目);AIGENDRUG Co., Ltd.(爱真药物有限公司);BK21 FOUR Intelligence Computing, Seoul National University(首尔国立大学BK21四号智能计算);Interdisciplinary Program in Artificial Intelligence, Seoul National University(首尔国立大学人工智能跨学科项目);Department of Artificial Intelligence, Inha University(仁荷大学人工智能系)
本文讨论了情感分类的问题。该任务包括预测最能描述短视频剪辑中包含的情绪的情绪标签(从一组可能的标签中选取)。建立在一个标准的框架-躺在描述视频的音频和视觉特征使用的监督分类器来推断标签-本文研究了几个新的方向。首先,提出了基于二维和三维卷积神经网络的改进人脸描述子。其次,本文探讨了几种融合方法,时间和多模态,包括一种新的分层方法相结合的功能和分数。此外,我们仔细审查了流水线的不同阶段,并设计了一个适应任务的CNN架构;这一点很重要,因为与问题的难度相比,训练集的大小很小,使得泛化困难。该模型在2017年的Emotion in the Wild挑战赛中排名第四,准确率为58.8%。