CARD：诊断视觉语言模型中从信念到行动的路由故障
CARD: Diagnosing Belief to Action Routing Failures in Vision Language Models
浏览论文内容
中文总结 AI 辅助
本文提出跨轴路由诊断（CARD）方法，在新型协作网格世界基准Relay Chain上诊断发现视觉语言模型（VLMs）存在未将信念表征纳入下一次行动预测的关键路由故障，相关成果为改进VLMs的信念-行动路由提供了依据。
中文摘要 AI 辅助
线性探测与激活调控已揭示，视觉语言模型（VLMs）内部表征着智能体的信念、知识与意图等心智状态，但这些表征是否及如何沿这些轴被下游预测使用尚不明确。为填补该缺口，本文提出跨轴路由诊断（CARD）方法，其沿某一轴调控激活，同时测量另一轴预测的响应。将该方法应用于开源权重VLMs及本文提出的新型协作网格世界基准Relay Chain上，研究人员诊断出一种关键路由故障：模型未能将信念表征纳入下一次行动预测，导致其合作伙伴的宝贵信息未被利用。
英文摘要
Linear probes and activation steering have uncovered that vision-language models (VLMs) internally represent mental states such as agents' beliefs, knowledge, and intentions. However, it is unclear whether and how these representations are used by downstream predictions along these axes. To close this gap, we introduce Cross-Axis Routing Diagnostic (CARD), which steers activations along one axis while measuring the response of a different axis's prediction. Applied to open-weight VLMs on Relay Chain -- a new cooperative grid-world benchmark we propose -- we diagnose a critical routing failure: models fail to incorporate belief representations into their next action prediction, effectively leaving valuable information about their partners unused.
发表机构
- Universität Stuttgart（斯图加特大学）
机构由 AI 辅助整理，请以论文原文为准。
