arXiv 2602.138322026-02cs.CL

超越词语：通过心灵理论评估和弥合用户代理交互中的认知分歧

BASIS: Evaluating Belief-Aware Solutions in Interactive Settings beyond Instructions

Dept. of Comp. Sci. \& Tech., Institute for AI, Tsinghua University, Beijing, China；Institute for AI Industry Research (AIR), Tsinghua University, Beijing, China

Minyuan Ruan, Ziyue Wang, Kaiming Liu, Yunghwei Lai, Peng Li, Yang Liu

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出通过心灵理论评估和弥合用户代理交互中的认知分歧，通过强化学习提升模型对用户心理状态的推理能力，从而增强下游任务性能。

摘要：

大型语言模型（LLMs）发展迅速，并被广泛应用于通用和专业任务中，以帮助人类用户。然而，当意图和指令传达不明确时，它们仍然难以理解和回应真正的用户需求，导致主观用户信念与真实环境状态之间出现分歧。解决这种认知分歧需要心灵理论（ToM），但现有的LLM ToM评估主要集中在孤立的信念推理上，忽略了其在现实世界交互中的功能效用。为此，我们将LLM中的ToM正式化为一种认知分歧检测和解决的机制，并提出一个基准测试集\benchname，以评估模型在实践中如何协调用户信念和档案。在11个领先模型上的结果揭示了识别底层认知差距的重大限制，这会阻碍任务成功。为了弥合这一差距，我们进一步整理了一个基于轨迹的ToM数据集，将信念跟踪与任务相关状态推理联系起来。通过强化学习训练的模型在推理用户心理状态方面表现出一致的改进，从而提高了下游性能。我们的工作强调了ToM作为交互层面机制的实用价值，而非作为独立推理技能。

英文摘要：

Large language model (LLM) agents are increasingly capable of carrying out complex, long-horizon tasks, yet effective assistance requires more than faithfully executing user instructions. Users' requests reflect their current understanding of the problem, which may be incomplete, outdated, or incorrect. When this understanding diverges from the actual task state, literal instruction following can fail to accomplish the user's underlying goal. We introduce \textbf{BASIS} (\emph{\textbf{B}elief-\textbf{A}ware \textbf{S}olutions in \textbf{I}nteractive \textbf{S}ettings}), a benchmark that evaluates whether agents can infer the mental states underlying user requests, recognize belief-reality mismatches, and use this understanding to develop appropriate solutions. Rather than assessing mental-state inference solely through explicit questions, \textbf{BASIS} evaluates both the utility of such inferences and the quality of downstream solutions in user-agent interactions. Evaluation results show that current models often fail to account for users' mental states when generating solutions, and that solution quality is consistently higher when such information is explicitly elicited and made available during problem solving. We further investigate the practical value of mental-state consideration through comparisons with generic Chain-of-Thought prompting, controlled interventions on mental-state information, targeted training, and multi-agent collaboration experiments under partial observability. Together, these studies highlight the importance of not merely inferring others' mental states, but also using those understandings to provide grounded, goal-directed assistance.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文