Interest makes one hold her attention on the object of interest. Automatic recognition of interest has numerous applications in human-computer interaction. In this paper, we study the facial expressions associated with interest and its underlying and closely related components, namely, curiosity, coping potential, novelty and complexity. To this end, we conducted an experiment in which participants watched images and micro-videos while a front-facing camera recorded their expressions. After watching each item they self-reported their level of interest, curiosity, coping potential and perceived novelty and complexity. Using an automated method, we tracked facial action units (AU) and studied the relationship between the presence of facial movements with interest and its related components. We then tracked the facial landmarks, e.g., corners of lips, and extracted features from each response. We trained random forests regression models to detect the level of interest, curiosity, and appraisals. We found a large difference between the way people report and react to interesting visual content. The expressions in response to images and micro-videos were not always pronounced depending on the participants. This makes the direct detection of interest from facial expressions a challenging problem. With this work, for the first time, we demonstrate the feasibility of detecting cognitive appraisals from facial expressions which will open the door for appraisal-driven emotion recognition methods.
Feasibility of Detecting Cognitive Impairment and Psychological Well-being among Older Adults Using Facial, Acoustic, Linguistic, and Cardiovascular Patterns Derived from Remote Conversations
Department of Computer Science, Emory University(计算机科学系,埃默里大学);The Wallace H. Coulter Department of Biomedical Engineering, Emory University and Georgia Institute of Technology(生物医学工程系,埃默里大学和佐治亚理工学院);Department of Systems and Biomedical Engineering, Cairo University(系统与生物医学工程系,开罗大学);Department of Biomedical Informatics, Emory University(生物医学信息学系,埃默里大学);Department of Neurology, Massachusetts General Hospital, Harvard Medical School(神经病学系,麻省总医院,哈佛医学院);Department of Psychiatry and Behavioral Sciences, Emory University(精神病学与行为科学系,埃默里大学)
Facial-Expression-Aware Prompting for Empathetic LLM Tutoring
Neurosciences Graduate Program, University of California San Diego(加州大学圣地亚哥分校神经科学研究生项目);Department of Cognitive Science, University of California San Diego(加州大学圣地亚哥分校认知科学系);Department of Computer Science and Engineering, University of California San Diego(加州大学圣地亚哥分校计算机科学与工程系);Department of Electrical and Computer Engineering, University of California San Diego(加州大学圣地亚哥分校电气与计算机工程系);Department of Mathematics, University of California San Diego(加州大学圣地亚哥分校数学系);Face the FACS;Halıcıoğlu Data Science Institute, University of California San Diego(Halıcıoğlu数据科学研究所)