Improved Emotional Alignment of AI and Humans: Human Ratings of Emotions Expressed by Stable Diffusion v1, DALL-E 2, and DALL-E 3
Delft University of Technology(代尔夫特理工大学);Delft Institute of Positive Design(代尔夫特积极设计研究所);Department of Human-Centered Design(以人为本设计系);Playpower Labs Pvt Ltd(Playpower实验室私人有限公司);School of Design(设计学院);Hong Kong Polytechnic University(香港理工大学);Department of Psychology(心理学系);Utrecht University(乌得勒支大学)
James Derek Lomas, Willem van der Maden, Sohhom Bandyopadhyay, Giovanni Lion, Nirmal Patel, Gyanesh Jain, Yanna Litowsky, Haian Xue, Pieter Desmet
Generative AI systems are increasingly capable of expressing emotions via text and imagery. Effective emotional expression will likely play a major role in the efficacy of AI systems -- particularly those designed to support human mental health and wellbeing. This motivates our present research to better understand the alignment of AI expressed emotions with the human perception of emotions. When AI tries to express a particular emotion, how might we assess whether they are successful? To answer this question, we designed a survey to measure the alignment between emotions expressed by generative AI and human perceptions. Three generative image models (DALL-E 2, DALL-E 3 and Stable Diffusion v1) were used to generate 240 examples of images, each of which was based on a prompt designed to express five positive and five negative emotions across both humans and robots. 24 participants recruited from the Prolific website rated the alignment of AI-generated emotional expressions with a text prompt used to generate the emotion (i.e., "A robot expressing the emotion amusement"). The results of our evaluation suggest that generative AI models are indeed capable of producing emotional expressions that are well-aligned with a range of human emotions; however, we show that the alignment significantly depends upon the AI model used and the emotion itself. We analyze variations in the performance of these systems to identify gaps for future improvement. We conclude with a discussion of the implications for future AI systems designed to support mental health and wellbeing.
Visual Stereotypes of Autism Spectrum in Janus-Pro-7B, DALL-E, Stable Diffusion, SDXL, FLUX, and Midjourney
Maria Curie-Skłodowska University(玛丽·居里大学);IDEAS Research Institute(IDEAS研究机构);Adam Mickiewicz University(亚当·密茨凯维奇大学);AMU Center for Artificial Intelligence(AMU人工智能中心);Poznań University of Medical Sciences(波兹南医科大学)
CARE: Commonsense-Aware Emotional Response Generation with Latent Concepts
Alibaba-NTU Singapore Joint Research Institute, Nanyang Technological University (NTU)(阿里巴巴-南洋理工大学新加坡联合研究院,南洋理工大学);Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly, NTU(南洋理工大学-不列颠哥伦比亚大学老年人积极生活联合卓越研究中心,南洋理工大学);School of Electrical and Electronic Engineering, NTU(南洋理工大学电气与电子工程学院);Alibaba Group(阿里巴巴集团)
Regressor-Guided Image Editing Shifts Emotion and Disengagement Timing in Social Media
Eastern Switzerland University of Applied Sciences \& ETH Z\"urich St.Gallen Switzerland;ETH Z\"urich Z\"urich Switzerland;The Hong Kong University of Science;Eastern Switzerland University of Applied Sciences \& ETH Z\"urich;ETH Z\"urich
Correspondence of high-dimensional emotion structures elicited by video clips between humans and Multimodal LLMs
The University of Tokyo, Graduate School of Arts and Sciences(东京大学艺术与科学研究生院);Center for Information and Neural Networks (CiNet), National Institute of Information and Communications Technology(信息与神经网络中心(CiNet),信息与通信技术国家研究所);The University of Osaka, Graduate School of Frontier Biosciences(大阪大学前沿生命科学研究生院);The University of Tokyo, Faculty of Engineering(东京大学工学部);The University of Osaka, Graduate School of Engineering Science(大阪大学工学研究院);The University of Osaka, Graduate School of Medicine(大阪大学医学研究院)
Bridging the behavior-neural gap: A multimodal AI reveals the brain's geometry of emotion more accurately than human self-reports
State Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Institute of Automation, Chinese Academy of Sciences(脑认知与脑启发智能技术重点实验室,自动化研究所,中国科学院);School of Artificial Intelligence, University of Chinese Academy of Sciences(人工智能学院,中国科学院大学);School of Future Technology, University of Chinese Academy of Sciences(未来技术学院,中国科学院大学);State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University(认知神经科学与学习国家重点实验室,北京师范大学)