语言模型智能体中目标导向性的行为与表征评估
A Behavioural and Representational Evaluation of Goal-Directedness in Language Model Agents
AI总结:
本文提出一种结合行为评估与内部表征可解释性分析的目标导向性评估框架,并以LLM智能体在2D网格世界中的导航为例,验证了其行为与表征的一致性。
摘要:
理解智能体的目标有助于解释和预测其行为,但目前尚无可靠的方法来归因智能系统的目标。我们提出一个评估目标导向性的框架,该框架将行为评估与基于可解释性的模型内部表征分析相结合。作为案例研究,我们考察了一个在二维网格世界中导航至目标状态的LLM智能体。在行为上,我们评估智能体在不同网格大小、障碍物密度和目标结构下的最优策略,发现其性能随任务难度扩展,同时对保持难度的变换和多目标结构具有鲁棒性。然后,我们使用探测方法解码环境及多步行动计划的内部表征。我们发现,LLM智能体非线性地编码了一个粗略的空间地图,保留了关于其位置和目标位置的任务相关近似线索;其行动与这些内部表征大致一致;推理过程重新组织这些表征,从空间线索转向即时行动选择。我们的研究结果支持这样的观点:除了行为评估之外,还需要内省检查来表征智能体如何表示和追求其目标。
英文摘要:
Understanding an agent's goals helps explain and predict its behaviour, yet there is no established methodology for reliably attributing goals to agentic systems. We propose a framework for evaluating goal-directedness that integrates behavioural evaluation with interpretability-based analyses of models' internal representations. As a case study, we examine an LLM agent navigating a 2D grid world towards a goal state. Behaviourally, we evaluate the agent against optimal policies across varying grid sizes, obstacle densities, and goal structures, finding that performance scales with task difficulty while remaining robust to difficulty-preserving transformations and multi-goal structures. We then use probing methods to decode internal representations of the environment and multi-step action plans. We find that the LLM agent non-linearly encodes a coarse spatial map, preserving approximate task-relevant cues about its position and the goal location; that its actions are broadly consistent with these internal representations; and that reasoning reorganises them, shifting from spatial cues towards immediate action selection. Our findings support the view that introspective examination is required beyond behavioural evaluations to characterise how agents represent and pursue their objectives.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。