TactfulToM：大语言模型是否具备理解善意的谎言的心智理论能力？
TactfulToM: Do LLMs Have the Theory of Mind Ability to Understand White Lies?
AI总结：
本文提出TactfulToM基准，评估大语言模型理解善意谎言及其亲社会动机的心智理论能力，发现最先进模型表现远低于人类。
摘要：
虽然近期研究探讨了大语言模型（LLMs）在心智理论（ToM）推理任务上的表现，但对需要更细微社会背景的心智理论能力（如善意的谎言）的研究仍然有限。我们引入了TactfulToM，这是一个新颖的英文基准，旨在评估大语言模型在真实生活对话中理解善意的谎言并推理其背后亲社会动机的能力，特别是当这些谎言被用来顾及他人感受和维护社会和谐时。我们的基准通过一个多阶段的人机协作流程生成，其中大语言模型将人工设计的种子故事扩展为对话，以保持参与者之间的信息不对称，这对于真实的善意的谎言至关重要。我们表明，TactfulToM对最先进的模型具有挑战性，它们的表现显著低于人类，揭示了它们在完全理解真正理解善意的谎言所需的心智理论推理方面的不足。
英文摘要：
While recent studies explore Large Language Models' (LLMs) performance on Theory of Mind (ToM) reasoning tasks, research on ToM abilities that require more nuanced social context is limited, such as white lies. We introduce TactfulToM, a novel English benchmark designed to evaluate LLMs' ability to understand white lies within real-life conversations and reason about prosocial motivations behind them, particularly when they are used to spare others' feelings and maintain social harmony. Our benchmark is generated through a multi-stage human-in-the-loop pipeline where LLMs expand manually designed seed stories into conversations to maintain the information asymmetry between participants necessary for authentic white lies. We show that TactfulToM is challenging for state-of-the-art models, which perform substantially below humans, revealing shortcomings in their ability to fully comprehend the ToM reasoning that enables true understanding of white lies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
