arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.09145cs.CY
收藏

你的脸红出卖了你:利用远程光电容积描记术与热成像检测隐藏的心理状态

Your blush gives you away: detecting hidden mental states with remote photoplethysmography and thermal imaging

Ivan Liu, Fangyuan Liu, Qi Zhong, Fei Ma, Shiguang Ni

AI总结:

本研究融合远程光电容积描记术与热成像,通过90名参与者的认知压力与道德提升感诱发实验,证明多模态早期融合策略可显著提升隐藏心理状态的检测准确率。

摘要:

多模态情绪识别技术对于评估心理状态日益重要。然而,基于图像的方法往往主要关注明显的外在视觉线索,并常常忽视更细微的心理状态变化。心理生理学研究表明,心率(HR)和皮肤温度能够有效检测自主神经系统(ANS)活动,从而揭示这些细微变化。然而,传统的心率测量工具通常成本更高且便携性较差,而皮肤温度分析通常需要大量人工处理。远程光电容积描记术(remote-PPG)和自动热成像感兴趣区域(ROI)检测算法的进展已被开发用于解决这些问题,但它们在实际应用中的准确性仍然有限。本研究旨在通过将r-PPG与热成像相结合来弥合这一差距,以提升预测性能。九十名参与者完成了一份20分钟的问卷以诱发认知压力,随后观看一部旨在引发道德提升感的影片。结果表明,r-PPG与热成像的结合能够有效检测情绪转变。仅使用r-PPG时,通过支持向量机(SVM)确定,对认知压力的预测准确率为77%,对道德提升感的预测准确率为61%。仅使用热成像时,利用随机森林(RF)算法,对认知压力实现了79%的准确率,对道德提升感实现了78%的准确率。对这些模态进行早期融合的策略显著提高了准确率,使用RF对认知压力达到87%,对道德提升感达到83%。进一步分析利用统计指标和包括SHAP在内的可解释机器学习方法,突出了关键特征,并阐明了心脏反应与面部温度变化之间的关系。值得注意的是,观察到尽管热成像在单模态分析中具有更高的预测准确率,但源自r-PPG模型的心血管特征在数据融合中具有更显著的影响。

英文摘要:

Multimodal emotion recognition techniques are increasingly essential for assessing mental states. Image-based methods, however, tend to focus predominantly on overt visual cues and often overlook subtler mental state changes. Psychophysiological research has demonstrated that HR and skin temperature are effective in detecting ANS activities, thereby revealing these subtle changes. However, traditional HR tools are generally more costly and less portable, while skin temperature analysis usually necessitates extensive manual processing. Advances in remote-PPG and automatic thermal ROI detection algorithms have been developed to address these issues, yet their accuracy in practical applications remains limited. This study aims to bridge this gap by integrating r-PPG with thermal imaging to enhance prediction performance. Ninety participants completed a 20-minute questionnaire to induce cognitive stress, followed by watching a film aimed at eliciting moral elevation. The results demonstrate that the combination of r-PPG and thermal imaging effectively detects emotional shifts. Using r-PPG alone, the prediction accuracy was 77% for cognitive stress and 61% for moral elevation, as determined by SVM. Thermal imaging alone achieved 79% accuracy for cognitive stress and 78% for moral elevation, utilizing a RF algorithm. An early fusion strategy of these modalities significantly improved accuracies, achieving 87% for cognitive stress and 83% for moral elevation using RF. Further analysis, which utilized statistical metrics and explainable machine learning methods including SHAP, highlighted key features and clarified the relationship between cardiac responses and facial temperature variations. Notably, it was observed that cardiovascular features derived from r-PPG models had a more pronounced influence in data fusion, despite thermal imaging's higher predictive accuracy in unimodal analysis.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑