用于过程级社会影响力评估的认知世界模型
Cognitive World Model for Progressive BDI/E Trajectory Evaluation of Conversational Agents
AI总结：
提出认知世界模型(CogWM)，通过联合预测BDI/E认知状态和用户话语，实现从终端判断到过程跟踪的社会影响力对话评估，在四个场景中达到77.6%情感准确率。
摘要：
社会影响力对话通过改变内部认知状态来改变用户行为。核心评估问题是用户的信念、愿望、意图和情绪在对话过程中是否发生可测量的变化，这是一个面向过程的标准，表面级别的文本指标（BLEU/ROUGE）或单一评分的LLM判断都无法捕捉。我们提出\textbf{认知世界模型（CogWM）}，一个基于LLM的用户模型，将多轮对话评估从“用户说了什么”重新定义为“用户内部认知状态如何演变”。CogWM联合预测BDI/E认知状态和用户话语，既作为用户模拟器也作为评估平台，使用三层评估框架，涵盖轮次级保真度、轨迹级状态动态和任务级复合评分。通过我们的\textbf{摘要与分配（SaA）}标注流水线在四个社会影响力场景的150,454个用户轮次样本上训练，CogWM实现了77.6%的情感准确率（是GPT-5.5的2.1倍）。在3600次多智能体判别试验中，它通过认知影响力区分了六个商业智能体，其中Llama-4-Scout排名第一（CTS +0.233）。CogWM将社会影响力对话评估从终端判断转向过程跟踪。我们已经发布了我们的代码\footnote{\scriptsize Code: https://github.com/lucianma05-create/CogWM}和模型\footnote{Model: https://www.modelscope.cn/models/LucianMa/CogWM-14B}。
英文摘要：
As LLM-based conversational agents advance toward increasingly open-ended and interaction-intensive scenarios, task completion alone provides an incomplete assessment of their effectiveness. The evolution of users' internal states, including beliefs, desires, intentions, and emotions (BDI/E), serves as an intermediate signal connecting agent behaviors with interaction outcomes and reflects how conversational strategies shape users during multi-turn interactions. However, existing evaluation paradigms primarily focus on surface-level responses or final outcomes, providing limited insight into the underlying cognitive processes. This limitation makes it difficult to diagnose why agents succeed or fail and to optimize their interaction strategies. To address this challenge, we propose Cognitive World Model (CogWM), an LLM-based cognitive user model that jointly models users' BDI/E states and corresponding responses, enabling explicit cognitive trajectory tracking. Trained on 150K user-turn samples with Qwen3-14B, CogWM achieves superior performance over existing user simulation baselines in both response fidelity and cognitive state understanding. Interactions with six state-of-the-art LLMs demonstrate that CogWM enables progressive comparison of agents through cognitive trajectories, revealing distinct agent patterns and complementary relationships between cognitive evolution and behavioral outcomes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
