Affective Feedback Synthesis Towards Multimodal Text and Image Data
Indian Institute of Technology Roorkee(印度理工学院鲁尔基分校);Indian Institute of Technology Hyderabad(印度理工学院海得拉巴分校);National Institute of Technology Karnataka(卡纳塔克邦国家理工学院)
In this paper, we have defined a novel task of affective feedback synthesis that deals with generating feedback for input text & corresponding image in a similar way as humans respond towards the multimodal data. A feedback synthesis system has been proposed and trained using ground-truth human comments along with image-text input. We have also constructed a large-scale dataset consisting of image, text, Twitter user comments, and the number of likes for the comments by crawling the news articles through Twitter feeds. The proposed system extracts textual features using a transformer-based textual encoder while the visual features have been extracted using a Faster region-based convolutional neural networks model. The textual and visual features have been concatenated to construct the multimodal features using which the decoder synthesizes the feedback. We have compared the results of the proposed system with the baseline models using quantitative and qualitative measures. The generated feedbacks have been analyzed using automatic and human evaluation. They have been found to be semantically similar to the ground-truth comments and relevant to the given text-image input.
OmniOPSD: Rationale-Privileged On-Policy Self-Distillation for Affective Computing
Shenzhen University(深圳大学);Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)(广东省人工智能与数字经济实验室(深圳));Tsinghua University(清华大学);Shanghai Jiao Tong University(上海交通大学);University of Science and Technology of China(中国科学技术大学);Shenzhen Technology University(深圳技术大学);Tongji University(同济大学);Huawei(华为)
SGED: A Benchmark dataset for Performance Evaluation of Spiking Gesture Emotion Recognition
Shandong Massive Information Technology Research Institute(山东海量信息技术研究院);State Key Laboratory of High-end Server & Storage Technology(高端服务器与存储技术国家重点实验室);Inspur (Beijing) Electronic Information Industry Co., Ltd.(浪潮(北京)电子信息产业有限公司);Inspur Electronic Information Industry Co., Ltd.(浪潮电子信息产业股份有限公司);Intel Labs China(英特尔中国研究院)
From Affect to Complex Behavior: Advancing Multimodal Human-Centered AI at the 10th ABAW Workshop & Competition
Queen Mary University of London(伦敦皇后玛丽大学);Hume AI;Google Deepmind(谷歌DeepMind);Imperial College London(伦敦帝国理工学院);Cogitat;LIVIA;ILLS;ETS Montreal(蒙特利尔ETS);Concordia University(Concordia大学);Xi’an Jiaotong University(西安交通大学)
Modeling emotion in complex stories: the Stanford Emotional Narratives Dataset
National University of Singapore(新加坡国立大学);A*STAR Artificial Intelligence Initiative, Agency for Science, Technology and Research(新加坡科技研究局人工智能计划);Stanford University(斯坦福大学);Massachusetts Institute of Technology(麻省理工学院);University of Oregon(俄勒冈大学)
Multimodal Emotion Recognition with Large Language Models
Tsinghua University(清华大学);Harbin Institute of Technology, Weihai(哈尔滨工业大学(威海));Academy of Cyber(网络空间安全学院);College of Computer Science, Nankai University(南开大学计算机科学学院)
Affective Computing for Large-Scale Heterogeneous Multimedia Data: A Survey
University of California, Berkeley(加州大学伯克利分校);University of Science and Technology of China(中国科学技术大学);University of Southern California(南加州大学);IBM Research AI(IBM人工智能研究院);Rensselaer Polytechnic Institute(伦斯勒理工学院)
Unify-Agent: A Unified Multimodal Agent for World-Grounded Image Synthesis
University of California, Los Angeles(加州大学洛杉矶分校);Tencent Hunyuan(腾讯混元);The Chinese University of Hong Kong(香港中文大学);The Hong Kong University of Science and Technology(香港科技大学)
Foley-Omni: A Unified Multimodal Generation Model from Task-Level Audio Synthesis to Complete Video Soundtrack Generation
School of Intelligence Science and Technology, Nanjing University(南京大学智能科学与技术学院);Video Rebirth;Shanghai Jiao Tong University(上海交通大学);Beijing Jiaotong University(北京交通大学);Shanghai AI Laboratory(上海人工智能实验室)
Bridging the behavior-neural gap: A multimodal AI reveals the brain's geometry of emotion more accurately than human self-reports
State Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Institute of Automation, Chinese Academy of Sciences(脑认知与脑启发智能技术重点实验室,自动化研究所,中国科学院);School of Artificial Intelligence, University of Chinese Academy of Sciences(人工智能学院,中国科学院大学);School of Future Technology, University of Chinese Academy of Sciences(未来技术学院,中国科学院大学);State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University(认知神经科学与学习国家重点实验室,北京师范大学)