存在但未被记住:审视冻结的视觉语言动作模型如何编码、部署和引导视觉历史
Present but Not Remembered: Auditing How Frozen VLAs Encode, Deploy, and Steer Visual History
AI总结:
研究通过线性探测和因果互换干预,探讨冻结的视觉语言动作模型对视觉历史的编码、部署和引导方式,发现模型对历史的处理存在三部分解离,并引入免训练的时间部署审计区分不同机制。
摘要:
一个冻结的视觉语言动作模型(VLA)在每个决策步骤接收最近的观察结果,但先前工作侧重于添加记忆而非询问现有历史如何被表示和使用。我们使用层解析线性探测和因果互换干预研究这个时间轴。我们发现了三部分解离。结果表明可操纵性取决于历史的部署方式而非是否被编码。VLA不会忘记过去;它们很大程度上未能将其表示为与当前不同的信息。我们的发现表明未来的记忆增强应注入过去独有的信息而非仅仅更多的历史。
英文摘要:
A frozen vision-language-action model (VLA) receives recent observations at every decision step, yet prior work has focused on adding memory rather than asking how existing history is represented and used. We study this temporal axis using layer-resolved linear probing and causal interchange interventions across three VLAs from two architecture families. We find a three-part dissociation. First, past-frame content remains linearly decodable throughout the network. Second, information unique to history beyond the current frame is nearly absent, indicating that stored history is largely a redundant copy of the present. Third, history is causally deployed only when the current frame is heavily degraded, while the action readout progressively loses dependence on history through the network. Although all models encode history similarly, their deployment strategies differ: under the same occlusion, one architecture increasingly relies on history as a fallback, whereas the other relies on it less. We further introduce a training-free temporal deployment audit that distinguishes these regimes. In the fallback regime, re-injecting history neither repairs occlusion nor disambiguates actions, confirming the redundancy of the stored representation. In the other regime, the same intervention reliably steers the predicted action toward the donor history. These results show that steerability depends on how history is deployed rather than whether it is encoded. VLAs do not forget the past; they largely fail to represent it as information distinct from the present. Our findings suggest that future memory augmentation should inject information unique to the past rather than simply more history.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。