可恨的人还是可恨的模型？调查人格角色在大语言模型仇恨言论检测中的作用
Hateful Person or Hateful Model? Investigating the Role of Personas in Hate Speech Detection by Large Language Models
AI总结：
本文研究 MBTI 人格角色提示对 LLM 仇恨言论分类的影响，通过人工调查与四模型、三数据集实验揭示角色导致的标签分歧和 logit 偏差。
摘要：
仇恨言论检测是一项具有社会敏感性且本质上主观的任务，其判断往往会因个人特征而异。尽管先前工作已经考察了社会人口统计因素如何影响标注，但人格特征对大语言模型（LLM）的影响在很大程度上仍未得到探索。本文首次全面研究了人格角色提示在仇恨言论分类中的作用，重点关注基于 MBTI 的特征。一项人工标注调查证实，MBTI 维度会显著影响标注行为。将此扩展到 LLM，我们用 MBTI 人格角色提示四个开源模型，并在三个仇恨言论数据集上评估其输出。分析揭示了由人格角色驱动的显著差异，包括与真实标签不一致、不同人格角色之间的分歧，以及 logit 层面的偏差。这些发现凸显了在基于 LLM 的标注工作流中谨慎定义人格角色提示的必要性，并对公平性以及与人类价值观对齐具有启示意义。
英文摘要：
Hate speech detection is a socially sensitive and inherently subjective task, with judgments often varying based on personal traits. While prior work has examined how socio-demographic factors influence annotation, the impact of personality traits on Large Language Models (LLMs) remains largely unexplored. In this paper, we present the first comprehensive study on the role of persona prompts in hate speech classification, focusing on MBTI-based traits. A human annotation survey confirms that MBTI dimensions significantly affect labeling behavior. Extending this to LLMs, we prompt four open-source models with MBTI personas and evaluate their outputs across three hate speech datasets. Our analysis uncovers substantial persona-driven variation, including inconsistencies with ground truth, inter-persona disagreement, and logit-level biases. These findings highlight the need to carefully define persona prompts in LLM-based annotation workflows, with implications for fairness and alignment with human values.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
