PHAnToM：基于人设的提示对大型语言模型中的心智理论推理具有影响
PHAnToM: Persona-based Prompting Has An Effect on Theory-of-Mind Reasoning in Large Language Models
AI总结：
本文实证研究角色扮演式提示对LLMs心智理论推理的影响，提出社会动机性提示差异是导致性能差异的机制，并警示采用特定人设可能引发社会认知推理错误。
摘要：
大型语言模型（LLMs）在自然语言推理中的应用呈现出好坏参半的结果：在较简单的分类任务中，它们有时能达到甚至超越人类表现，但在社会认知推理这一人类天然擅长的领域却表现挣扎。这些差异被归因于多种因素，例如提示方式的变化以及所使用的具体LLMs。然而，此前的研究并未给出确定性的原因，也未建立清晰的机制。在本研究中，我们实证评估了角色扮演式提示如何影响心智理论（ToM）推理能力。以心理学理论为基础，我们提出如下机制：除了推理任务本身固有的复杂度差异之外，性能差异还源于具有社会动机的提示方式差异。在一个通过角色扮演进行提示工程已成为将LLMs适配到新情境的典型方法的时代，我们的研究倡导保持谨慎，因为采用特定人设的模型可能会在社会认知推理中产生错误。
英文摘要：
The use of LLMs in natural language reasoning has shown mixed results, sometimes rivaling or even surpassing human performance in simpler classification tasks while struggling with social-cognitive reasoning, a domain where humans naturally excel. These differences have been attributed to many factors, such as variations in prompting and the specific LLMs used. However, no reasons appear conclusive, and no clear mechanisms have been established in prior work. In this study, we empirically evaluate how role-playing prompting influences Theory-of-Mind (ToM) reasoning capabilities. Grounding our rsearch in psychological theory, we propose the mechanism that, beyond the inherent variance in the complexity of reasoning tasks, performance differences arise because of socially-motivated prompting differences. In an era where prompt engineering with role-play is a typical approach to adapt LLMs to new contexts, our research advocates caution as models that adopt specific personas might potentially result in errors in social-cognitive reasoning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
