探索动力学差距：通过动作-结果反馈进行测试时策略自适应
Scouting the Dynamics Gap: Test-Time Policy Adaptation via Action-Outcome Feedback
浏览论文内容
中文总结 AI 辅助
针对预训练机器人策略在部署时需适应未知动力学的问题，提出SCOUT元学习框架，通过共享信念潜空间耦合动作预测与前向模型，利用动作-结果反馈在线更新信念，加速模拟及现实中的自适应。
中文摘要 AI 辅助
虽然预训练的机器人策略在受控环境中展现出令人印象深刻的能力，但未观测到的物理属性和动力学要求这些策略在部署期间快速适应。现有的测试时自适应方法通常依赖稀疏的标量奖励，未能利用物理交互过程中环境提供的丰富几何和动力学反馈。为应对这一挑战，我们提出SCOUT，一种动力学感知的元学习框架，使操作策略能够通过持续修正其对环境动力学的内部信念来快速适应。我们的方法通过共享的信念潜空间将动作预测策略与前向动力学模型耦合。在元训练期间，内循环通过最小化针对观测到的动作结果的动力学预测误差来更新此共享信念潜变量，而外循环则优化网络以进行动作选择。在部署时，这种结构使智能体能够即时推断并适应未知的物理动力学。通过基于动作-结果不匹配更新其潜在信念，策略自动适应而不会冒灾难性遗忘的风险。我们证明SCOUT在模拟操作基准测试中显著加速了在线适应，并在现实世界中实现了稳健的模拟到现实迁移。项目网站可在此处找到：此https URL
英文摘要
While pretrained robotic policies exhibit impressive capabilities in controlled environments, unobserved physical properties and dynamics require these policies to rapidly adapt during deployment. Existing test-time adaptation methods typically rely on sparse scalar rewards, failing to exploit the rich geometric and dynamic feedback from the environment during physical interaction. To address this challenge, we propose SCOUT, a dynamics-aware meta-learning framework that enables manipulation policies to rapidly adapt by continuously revising their internal beliefs about environment dynamics. Our approach couples an action-prediction policy with a forward dynamics model via a shared belief latent space. During meta-training, an inner loop updates this shared belief latent by minimizing the dynamics prediction error against the observed action outcome, while the outer loop optimizes the network for action selection. At deployment, this structure allows the agent to infer and adapt to unknown physical dynamics on the fly. By updating its latent belief based on action-outcome mismatches, the policy automatically adapts without risking catastrophic forgetting. We demonstrate that SCOUT significantly accelerates online adaptation across simulated manipulation benchmarks and achieves robust sim-to-real transfer in the real world. Project webiste can be found here: https://liy1shu.github.io/SCOUT/
发表机构
- Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）
- Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）
机构由 AI 辅助整理，请以论文原文为准。
