arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.20323cs.ROcs.AI
收藏

PhysMem:为机器人操作扩展测试时的物理内存

PhysMem: Scaling Test-time Physical Memory for Robot Manipulation

Stanford University(斯坦福大学);UC San Diego(圣地亚哥大学)

Haoyang Li, Yang You, Hao Su, Leonidas Guibas

AI总结:

PhysMem通过测试时交互学习物理原理,提升机器人在不同环境下的物体操控能力,实验显示其在真实任务和模拟中均优于传统经验检索方法。

摘要:

可靠的物体操控需要理解随物体和环境变化的物理属性。视觉-语言模型(VLM)规划器可以一般性地推理摩擦和稳定性,但通常无法预测特定球在特定表面的滚动或特定石块的稳定性,而无需直接经验。我们提出了PhysMem,一种记忆框架,使VLM机器人规划器能够在测试时通过交互学习物理原理,而无需更新模型参数。系统记录经验、生成候选假设,并通过定向交互验证假设,然后将验证的知识推广以指导未来决策。关键设计选择是验证后再应用:系统通过新观察测试假设,而非直接应用检索经验,从而在物理条件变化时减少对先前经验的刚性依赖。我们评估了PhysMem在三个现实操控任务和四个VLM基础架构的模拟基准上的表现。在受控砖块插入任务中,原理性抽象实现了76%的成功率,相比直接经验检索的23%,现实实验显示在30分钟部署会话中持续改进。

英文摘要:

Reliable object manipulation requires understanding physical properties that vary across objects and environments. Vision-language model (VLM) planners can reason about friction and stability in general terms; however, they often cannot predict how a specific ball will roll on a particular surface or which stone will provide a stable foundation without direct experience. We present PhysMem, a memory framework that enables VLM robot planners to learn physical principles from interaction at test time, without updating model parameters. The system records experiences, generates candidate hypotheses, and verifies them through targeted interaction before promoting validated knowledge to guide future decisions. A central design choice is verification before application: the system tests hypotheses against new observations rather than applying retrieved experience directly, reducing rigid reliance on prior experience when physical conditions change. We evaluate PhysMem on three real-world manipulation tasks and simulation benchmarks across four VLM backbones. On a controlled brick insertion task, principled abstraction achieves 76% success compared to 23% for direct experience retrieval, and real-world experiments show consistent improvement over 30-minute deployment sessions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑