Recent years have witnessed remarkable progress in image generation task, where users can create visually astonishing images with high-quality. However, existing text-to-image diffusion models are proficient in generating concrete concepts (dogs) but encounter challenges with more abstract ones (emotions). Several efforts have been made to modify image emotions with color and style adjustments, facing limitations in effectively conveying emotions with fixed image contents. In this work, we introduce Emotional Image Content Generation (EICG), a new task to generate semantic-clear and emotion-faithful images given emotion categories. Specifically, we propose an emotion space and construct a mapping network to align it with the powerful Contrastive Language-Image Pre-training (CLIP) space, providing a concrete interpretation of abstract emotions. Attribute loss and emotion confidence are further proposed to ensure the semantic diversity and emotion fidelity of the generated images. Our method outperforms the state-of-the-art text-to-image approaches both quantitatively and qualitatively, where we derive three custom metrics, i.e., emotion accuracy, semantic clarity and semantic diversity. In addition to generation, our method can help emotion understanding and inspire emotional art design.
EmoGene: Audio-Driven Emotional 3D Talking-Head Generation
Khoury College of Computer Science, Northeastern University, USA(东北大学Khoury计算机科学学院);Department of Electrical and Computer Engineering, Northeastern University, USA(东北大学电气与计算机工程系)
EmoAttack: Emotion-to-Image Diffusion Models for Emotional Backdoor Generation
School of Software Engineering, Xi’an Jiaotong University(西安交通大学软件工程学院);Tongji University(同济大学);College of Computer Science, Nankai University(南开大学计算机科学学院)
EmoDiffusion: Enhancing Emotional 3D Facial Animation with Latent Diffusion Models
University of Chinese Academy of Sciences(中国科学院大学);University of Science and Technology Beijing(北京科技大学);The New Laboratory of Pattern Recognition (NLPR), State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所多模态人工智能系统国家重点实验室模式识别新实验室);The New Laboratory of Pattern Recognition (NLPR)(模式识别新实验室)
MUSE: Manipulating Unified Framework for Synthesizing Emotions in Images via Test-Time Optimization
VCIP & TMCC & DISSec, College of Computer Science, Nankai University(南开大学计算机学院);LIX, Ecole Polytechnique, IP Paris(巴黎高等理工学院);Key Lab of SCCI, Dalian University of Technology(大连理工大学SCCI重点实验室)
Towards Deeper Emotional Reflection: Crafting Affective Image Filters with Generative Priors
School of Artificial Intelligence, Beijing University of Posts and Telecommunications(北京邮电大学人工智能学院);Beijing Academy of Artificial Intelligence(北京人工智能研究院);State Key Laboratory of Multimedia Information Processing and National Engineering Research Center of Visual Technology, School of Computer Science, Peking University(北京大学多媒体信息处理国家重点实验室和视觉技术国家工程研究中心)
Nanjing University(南京大学);The Chinese University of Hong Kong(香港中文大学);The University of Sydney(悉尼大学);SenseTime Research(商汤科技研究院);Nanyang Technological University(南洋理工大学);Tsinghua University(清华大学)
EmoSpeaker: One-shot Fine-grained Emotion-Controlled Talking Face Generation
School of Computer Science and Technology, Xidian University(西安电子科技大学计算机科学与技术学院);Xi’an Key Laboratory of Big Data and Intelligent Vision, Xidian University(西安电子科技大学西安大数据与智能视觉重点实验室);Department of Computer and Information Science, University of Macau(澳门大学计算机与信息科学系)
Enriching Multimodal Sentiment Analysis through Textual Emotional Descriptions of Visual-Audio Content
Tianjin University(天津大学);Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)(广东人工智能与数字经济实验室(深圳));Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院)
EmoTalk: Speech-Driven Emotional Disentanglement for 3D Face Animation
Renmin University of China(中国人民大学);Nanjing University of Science and Technology(南京理工大学);The Hong Kong University of Science and Technology(香港科技大学);Psyche AI Inc.(赛琪人工智能公司);Chinese Academy of Sciences(中国科学院);Tsinghua University(清华大学)
Doctoral Program in Electrical and Computer Engineering(电子与计算机工程博士项目);Faculty of Engineering, University of Porto(葡萄牙波尔图大学工程学院);Faculty of Science and Technology, University of Coimbra(科英布拉大学科学与技术学院);Frost School of Music, University of Miami(迈阿密大学弗罗斯特音乐学院);Faculty of Science and Technology, NOVA University Lisbon(里斯本NOVA大学科学与技术学院);ISEP, Polytechnic of Porto(波尔图理工学院ISEP)