发表机构
Hangzhou Normal University; Southeast University（杭州师范大学; 东南大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究针对异步视频面试人格评估中以文本为中心的方法忽略非言语线索的问题，提出EMMR框架，在OPVA和AVI-6数据集上提升了MAE、MSE和PCC，证明整合情绪线索可优化评估效果。
AI 中文摘要
异步视频面试（AVIs）已在人格评估中愈发普及。近期的大语言模型（LLMs）展现出从转录的面试应答中进行人格评估的潜力，但以文本为中心的方法可能会忽略视觉与音频模态传递的非言语行为线索，而这类线索与人格评估高度相关。尤其，情绪相关线索为理解与人格特质相关的候选人行为提供了重要的社会与情感证据。因此，我们提出EMMR（情绪介导多模态推理），这是一个基于多模态大语言模型（MLLMs）的异步视频面试人格评估两阶段框架。EMMR从多模态面试数据中提取情绪相关线索，并将其作为辅助的社会与行为证据，通过结构化推理融入人格评估。在OPVA和AVI-6两个异步视频面试数据集上的实验表明，与基线方法相比，EMMR降低了平均绝对误差（MAE）、均方误差（MSE）并提高了皮尔逊相关系数（PCC）。进一步分析显示，情绪线索的语义描述可提升人格评估效果，而其质量会影响人格评估的可靠性。这些结果表明，将情绪相关线索整合进多模态推理，是实现更具可解释性的、基于多模态大语言模型的异步视频面试人格评估的有前景方向。
英文摘要
Asynchronous Video Interviews (AVIs) have become increasingly popular for personality assessment. Recent large language models (LLMs) have shown potential for personality assessment from transcribed interview responses. However, text-centered methods may overlook non-verbal behavioral cues conveyed through visual and audio modalities, even though such cues are highly relevant to personality assessment. In particular, emotion-related cues provide important social and affective evidence for understanding candidates' behavior related to personality traits. Thus, we propose EMMR (Emotion-Mediated Multimodal Reasoning), a two-stage framework for MLLMs-based personality assessment for AVIs. EMMR extracts emotion-related cues from multimodal interview data and incorporates them into personality assessment through structured reasoning as auxiliary social and behavioral evidence. Experiments on two AVIs datasets, OPVA and AVI-6, show that EMMR improves MAE, MSE, and PCC compared with baselines. Further analysis indicates that semantic descriptions of emotion cues enhance personality assessment, while their quality affects personality assessment reliability. These results suggest that integrating emotion-related cues into multimodal reasoning is a promising direction for more interpretable MLLMs-based personality assessment in AVIs.
