7 pages, Badoy, W. and Teknomo, K. (2014) Q-Learning with Basic Emotions, Proceeding of the 7th IEEE International Conference Humanoid, Nanotechnology, Information Technology Communication and Control, Environment and Management (HNICEM) 12-16 November 2014 Hotel Centro, Puerto Princesa, Palawan, Philippines
Q-learning is a simple and powerful tool in solving dynamic problems where environments are unknown. It uses a balance of exploration and exploitation to find an optimal solution to the problem. In this paper, we propose using four basic emotions: joy, sadness, fear, and anger to influence a Qlearning agent. Simulations show that the proposed affective agent requires lesser number of steps to find the optimal path. We found when affective agent finds the optimal path, the ratio between exploration to exploitation gradually decreases, indicating lower total step count in the long run
University of Canberra（堪培拉大学）；Kuwait College of Science and Technology（科威特科技学院）
While Parkinson's disease (PD) is typically characterized by motor disorder, there is evidence of diminished emotion perception in PD patients. This study examines the utility of affective Electroencephalography (EEG) signals to understand emotional differences between PD vs Healthy Controls (HC), and for automated PD detection. Employing traditional machine learning and deep learning methods, we explore (a) dimensional and categorical emotion recognition, and (b) PD vs HC classification from emotional EEG signals. Our results reveal that PD patients comprehend arousal better than valence, and…
Affective Multimodal Agents with Proactive Knowledge Grounding for Emotionally Aligned Marketing Dialogue
Hunan Police Academy, Department of Criminal Investigation（湖南警察学院犯罪侦查系）；Business College, California State University（加州州立大学长滩分校商学院）；Northwestern University（西北大学）；AI Agent Lab, Vokram Group（Vokram集团AI代理实验室）；Beijing University of Technology（北京理工大学）；Zheng Zhou University of Light Industry（郑州轻工业大学）
A Comparative Study on Affective Cues in Text Embeddings Across Psychological Emotion Theories
Department of Electronics, Information, and Bioengineering, Polytechnic University of Milan（米兰理工学院电子、信息与生物工程系）；Multimedia Mining and Search Group, Institute of Computational Perception, Johannes Kepler University Linz（莱茵河畔林茨约翰·凯撒大学计算感知研究所多媒体挖掘与检索小组）；Department of Music Pedagogy, Nuremberg University of Music（纽伦堡音乐学院音乐教育系）；Human-centered AI Group, AI Lab, Linz Institute of Technology（林茨技术学院以人为本的人工智能小组）
E^2-LLM: Bridging Neural Signals and Interpretable Affective Analysis
Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)（广东人工智能与数字经济实验室）；Zhejiang University（浙江大学）；Tsinghua University（清华大学）；Harbin Institute of Technology（哈尔滨工业大学）；Eastern Institute of Technology（东方技术研究所）；Huawei（华为）
Adobe Research（奥多比研究院）；University of Warwick（华威大学）
We introduce the problem of learning affective correspondence between audio (music) and visual data (images). For this task, a music clip and an image are considered similar (having true correspondence) if they have similar emotion content. In order to estimate this crossmodal, emotion-centric similarity, we propose a deep neural network architecture that learns to project the data from the two modalities to a common representation space, and performs a binary classification task of predicting the affective correspondence (true or false). To facilitate the current study, we construct a large s…
Indian Institute of Technology Kanpur (IITK)（印度理工学院坎普尔分校（IITK））
Human use language not just to convey information but also to express their inner feelings and mental states. In this work, we adapt the state-of-the-art language generation models to generate affective (emotional) text. We posit a model capable of generating affect-driven and topic-focused sentences without losing grammatical correctness as the affect intensity increases. We propose to incorporate emotion as prior for the probabilistic state-of-the-art text generation model such as GPT-2. The model gives a user the flexibility to control the category and intensity of emotion as well as the to…
Columbia University（哥伦比亚大学）；Nankai University（南开大学）；Tsinghua University（清华大学）；University of California, Berkeley（加州大学伯克利分校）
Humans are emotional creatures. Multiple modalities are often involved when we express emotions, whether we do so explicitly (e.g., facial expression, speech) or implicitly (e.g., text, image). Enabling machines to have emotional intelligence, i.e., recognizing, interpreting, processing, and simulating emotions, is becoming increasingly important. In this tutorial, we discuss several key aspects of multi-modal emotion recognition (MER). We begin with a brief introduction on widely used emotion representation models and affective modalities. We then summarize existing emotion annotation strateg…
EmoMind: Decoding Affective Captions from Human Brain fMRI
Department of Biomedical Engineering（生物医学工程系）；Vanderbilt University（范德比大学）；Research Institute of Electrical Communication（电气通信研究所）；Tohoku University（东北大学）；University of Florida（佛罗里达大学）
King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学（KAUST））
Datasets that capture the connection between vision, language, and affection are limited, causing a lack of understanding of the emotional aspect of human intelligence. As a step in this direction, the ArtEmis dataset was recently introduced as a large-scale dataset of emotional reactions to images along with language explanations of these chosen emotions. We observed a significant emotional bias towards instance-rich emotions, making trained neural speakers less accurate in describing under-represented emotions. We show that collecting new data, in the same way, is not effective in mitigating…
University of Applied Sciences of the Grisons（格劳宾登应用科学大学）；MODUL Technology（MODUL科技公司）；MODUL University Vienna（维也纳模都尔大学）；webLyzard technology（webLyzard科技公司）
Sentic computing relies on well-defined affective models of different complexity - polarity to distinguish positive and negative sentiment, for example, or more nuanced models to capture expressions of human emotions. When used to measure communication success, even the most granular affective model combined with sophisticated machine learning approaches may not fully capture an organisation's strategic positioning goals. Such goals often deviate from the assumptions of standardised affective models. While certain emotions such as Joy and Trust typically represent desirable brand associations,…
We present a learning-based method for detecting real and fake deepfake multimedia content. To maximize information for learning, we extract and analyze the similarity between the two audio and visual modalities from within the same video. Additionally, we extract and compare affective cues corresponding to perceived emotion from the two modalities within a video to infer whether the input video is "real" or "fake". We propose a deep learning network, inspired by the Siamese network architecture and the triplet loss. To validate our model, we report the AUC metric on two large-scale deepfake d…
University of Southern California（南加州大学）；JD AI Research（京东人工智能研究院）
Representation learning for speech emotion recognition is challenging due to labeled data sparsity issue and lack of gold standard references. In addition, there is much variability from input speech signals, human subjective perception of the signals and emotion label ambiguity. In this paper, we propose a machine learning framework to obtain speech emotion representations by limiting the effect of speaker variability in the speech signals. Specifically, we propose to disentangle the speaker characteristics from emotion through an adversarial training network in order to better represent emot…
Samsung AI Center Cambridge（三星剑桥人工智能中心）；University of Nottingham（诺丁汉大学）
Temporal context is key to the recognition of expressions of emotion. Existing methods, that rely on recurrent or self-attention models to enforce temporal consistency, work on the feature level, ignoring the task-specific temporal dependencies, and fail to model context uncertainty. To alleviate these issues, we build upon the framework of Neural Processes to propose a method for apparent emotion recognition with three key novel components: (a) probabilistic contextual representation with a global latent variable model; (b) temporal context modelling using task-specific predictions in additio…
As a common way of emotion signaling via non-linguistic vocalizations, vocal burst (VB) plays an important role in daily social interaction. Understanding and modeling human vocal bursts are indispensable for developing robust and general artificial intelligence. Exploring computational approaches for understanding vocal bursts is attracting increasing research attention. In this work, we propose a hierarchical framework, based on chain regression models, for affective recognition from VBs, that explicitly considers multiple relationships: (i) between emotional states and diverse cultures; (ii…
National University of Singapore（新加坡国立大学）；Singapore University of Technology and Design（新加坡科技设计大学）；Goethe University Frankfurt（法兰克福大学）
Although media content is increasingly produced, distributed, and consumed in multiple combinations of modalities, how individual modalities contribute to the perceived emotion of a media item remains poorly understood. In this paper we present MusicVideos (MuVi), a novel dataset for affective multimedia content analysis to study how the auditory and visual modalities contribute to the perceived emotion of media. The data were collected by presenting music videos to participants in three conditions: music, visual, and audiovisual. Participants annotated the music videos for valence and arousal…
Body movements are an important communication medium through which affective states can be discerned. Movements that convey affect can also give machines life-like attributes and help to create a more engaging human-machine interaction. This paper presents an approach for automatic affective movement generation that makes use of two movement abstractions: 1) Laban movement analysis (LMA), and 2) hidden Markov modeling. The LMA provides a systematic tool for an abstract representation of the kinematic and expressive characteristics of movements. Given a desired motion path on which a target emo…
