人格感知的以人为中心多模态推理：一项新任务、数据集与基线
Personality-aware Human-centric Multimodal Reasoning: A New Task, Dataset and Baselines
AI总结：
本文提出人格感知的以人为中心多模态推理任务PHMR，构建含225个角色、1.2万样本的数据集并评估七类基线，证明人格信息能提升行为预测，还扩展提出T2以缓解人格标注缺失。
摘要：
人格特质、情绪和信念塑造着个体的行为选择与决策过程。然而，一方面，情感计算领域通常关注预测人格特质，却忽视了其在行为预测中的应用；另一方面，多模态推理任务强调对未来状态和行为的预测，却常常忽略纳入个体人格特质。在本工作中，我们提出一项名为 Personality-aware Human-centric Multimodal Reasoning（PHMR，人格感知的以人为中心多模态推理）的新任务（T1），目标是利用过去实例中的多模态信息预测特定个体的未来行为，同时整合人格因素。我们据此基于六部电视剧构建了一个新数据集，包含225个角色和1.2万个样本。为建立该任务的基准，我们提出七种基线方法：三种改编自相关任务，两种为预训练模型，两种为多模态大语言模型。实验结果表明，纳入人格特质可提升以人为中心的多模态推理性能。为进一步解决现实场景中人格标注缺失的问题，我们引入一项扩展任务，即 Personality-predicted Human-centric Multimodal Reasoning task（T2，人格预测型以人为中心多模态推理任务），并提供相应数据集和方法。我们将在 GitHub 上公开数据集和代码。
英文摘要：
Personality traits, emotions, and beliefs shape individuals' behavioral choices and decision-making processes. However, for one thing, the affective computing community normally focused on predicting personality traits but overlooks their application in behavior prediction. For another, the multimodal reasoning task emphasized the prediction of future states and behaviors but often neglected the incorporation of individual personality traits. In this work, we introduce a new task called Personality-aware Human-centric Multimodal Reasoning (PHMR) (T1), with the goal of forecasting the future behavior of a particular individual using multimodal information from past instances, while integrating personality factors. We accordingly construct a new dataset based on six television shows, encompassing 225 characters and 12k samples. To establish a benchmark for the task, we propose seven baseline methods: three adapted from related tasks, two pre-trained model, and two multimodal large language models. The experimental results demonstrate that incorporating personality traits enhances human-centric multimodal reasoning performance. To further solve the lack of personality annotation in real-life scenes, we introduce an extension task called Personality-predicted Human-centric Multimodal Reasoning task (T2) along with the corresponding dataset and method. We will make our dataset and code available on GitHub.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
