机器人策略的高效测试时自适应
Efficient Test-Time Adaptation For Robot Policies
AI总结：
针对机器人策略部署时运行条件变化导致失效的问题，提出TEMPO测试时自适应方法，在MuJoCo仿真和物理Unitree机器人上实现了最高80%的性能提升。
摘要：
用于机器人运动的强化学习策略在部署时，若运行条件发生变化（如地形崎岖或湿滑、负载改变、感知性能下降）可能失效，而在设备延迟和内存约束下，重新训练或手动重新校准通常不可行。我们研究机器人控制的测试时自适应（TTA）：在执行过程中以自监督方式在线调整预训练策略，无需访问奖励。我们的起点是一个轻量级基线，仅使用冻结的预训练评论者作为部署时性能替代指标来更新演员。在此基础上，我们提出TEMPO，一种测试时熵正则化、记忆引导的策略优化方法，可在严重偏移下稳定评论者驱动的更新。TEMPO结合了（i）对评论者预测分布的熵促进正则化，以减轻过度自信外推；（ii）受优先经验回放启发的基于分数的记忆，对低价值（“困难”）观测进行优先级排序，并应用重要性采样，将有限计算集中在最具信息性的部署条件上。在MuJoCo Playground中的Go1、Go2和G1机器人上，针对四种代表性偏移和多种架构，TEMPO取得了一致的性能提升，在单设备设置下最高可达80%。我们还在添加了小腿负载的物理Unitree Go2上进行了验证，完全设备端自适应在真实场景的多个 episode 中，性能优于未自适应的基线。
英文摘要：
Reinforcement-learning policies for robot locomotion can fail at deployment when operating conditions shift (e.g., rough or slippery terrain, payload changes, sensing degradation), and retraining or manual recalibration is often infeasible under on-device latency and memory constraints. We study test-time adaptation (TTA) for robotic control: adapting a pretrained policy online and in a self-supervised manner during execution, without access to rewards. Our starting point is a lightweight baseline that updates only the actor using a frozen pretrained critic as a deployment-time performance surrogate. Building on this, we introduce TEMPO, a test-time entropy-regularized, memory-guided policy optimization method that stabilizes critic-driven updates under severe shift. TEMPO combines (i) an entropy-promoting regularizer on the critic's predictive distribution to mitigate overconfident extrapolation, and (ii) a score-based memory, inspired by prioritized replay, that prioritizes low-value ("hard") observations and applies importance sampling to focus limited computation on the most informative deployment conditions. Across Go1, Go2, and G1 in MuJoCo Playground under four representative shifts and multiple architectures, TEMPO yields consistent gains, reaching up to 80% improvement in a single-device setting. We further validate on a physical Unitree Go2 with an added calf payload, where fully on-device adaptation improves performance over the non-adapted baseline across real-world episodes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
