arXiv 2412.125102024-12-17cs.CLcs.CY

大型语言模型能否更好地理解你？一个与人群特质对齐的MBTI人格检测数据集

Can Large Language Models Understand You Better? An MBTI Personality Detection Dataset Aligned with Population Traits

Harbin Institute of Technology（哈尔滨工业大学）；Central South University（中南大学）

Bohan Li, Jiannan Guan, Longxu Dou, Yunlong Feng, Dingzirui Wang, Yang Xu, Enbo Wang, Qiguang Chen, Bichen Wang, Xiao Xu, Yimeng Zhang, Libo Qin, Yanyan Zhao, Q… 展开作者

Bohan Li, Jiannan Guan, Longxu Dou, Yunlong Feng, Dingzirui Wang, Yang Xu, Enbo Wang, Qiguang Chen, Bichen Wang, Xiao Xu, Yimeng Zhang, Libo Qin, Yanyan Zhao, Qingfu Zhu, Wanxiang Che

备注：: Accepted by COLING 2025. 28 papges, 20 figures, 10 tables

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文构建了首个带软标签的MBTI人格检测数据集MBTIBench，解决错误标注并反映人群特质分布，验证了软标签优于硬标签，并揭示了LLM的极化预测问题。

摘要：

迈尔斯-布里格斯类型指标（MBTI）是最具影响力的人格理论之一，反映了个体在思维、情感和行为方面的差异。MBTI人格检测引起了相当多的研究兴趣，并多年来得到了显著发展。然而，这一任务往往过于乐观，因为它目前与人群人格特质的自然分布不太一致。具体而言，（1）现有数据集中的自我报告标签导致了错误标注问题，（2）硬标签无法捕捉人群人格分布的完整范围。在本文中，我们通过构建MBTIBench来优化这一任务，这是第一个在心理学家指导下，带有软标签的人工标注高质量MBTI人格检测数据集。针对第一个挑战，MBTIBench有效解决了错误标注问题，这些问题占数据的29.58%。针对第二个挑战，我们通过推导样本的极性倾向来估计软标签。获得的软标签证实了具有非极端人格特质的人更多。实验结果不仅突出了大型语言模型中的极化预测和偏见作为未来研究的关键方向，而且确认了软标签相比硬标签能为其他心理任务带来更多益处。代码和数据可在https://github.com/Personality-NLP/MbtiBench获取。

英文摘要：

The Myers-Briggs Type Indicator (MBTI) is one of the most influential personality theories reflecting individual differences in thinking, feeling, and behaving. MBTI personality detection has garnered considerable research interest and has evolved significantly over the years. However, this task tends to be overly optimistic, as it currently does not align well with the natural distribution of population personality traits. Specifically, (1) the self-reported labels in existing datasets result in incorrect labeling issues, and (2) the hard labels fail to capture the full range of population personality distributions. In this paper, we optimize the task by constructing MBTIBench, the first manually annotated high-quality MBTI personality detection dataset with soft labels, under the guidance of psychologists. As for the first challenge, MBTIBench effectively solves the incorrect labeling issues, which account for 29.58% of the data. As for the second challenge, we estimate soft labels by deriving the polarity tendency of samples. The obtained soft labels confirm that there are more people with non-extreme personality traits. Experimental results not only highlight the polarized predictions and biases in LLMs as key directions for future research, but also confirm that soft labels can provide more benefits to other psychological tasks than hard labels. The code and data are available at https://github.com/Personality-NLP/MbtiBench.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。