MorFiC：解决零样本四足机器人跨形态转移中的值校准问题
MorFiC: Fixing Value Miscalibration for Zero-Shot Quadruped Transfer
AI总结：
MorFiC通过共享策略实现跨形态四足机器人运动策略的零样本迁移，解决共享批评者平均不兼容价值目标导致的校准问题，提升稳定速度和运行距离。
摘要：
跨形态四足机器人运动策略的泛化仍是一个挑战。在单一机器人上训练的策略在部署到形态不同的机器人时往往会失效，迫使每个机器人重新训练。我们提出了MorFiC，一种使用单一共享策略进行零样本跨形态运动的强化学习方法。MorFiC解决了多形态演员-评论家训练中的关键失败模式：共享评论者倾向于在不同形态之间平均不兼容的价值目标，导致价值校准错误。为了解决这个问题，MorFiC通过形态感知调制来调节评论者，该调制由机器人物理和控制参数驱动，生成特定形态的价值估计，这些估计在共享网络中进行。通过在单一源机器人上进行训练，其中形态在仿真中随机化，MorFiC能够转移到未见过的机器人，并在多个目标上超越了形态条件化的PPO基线，通过提高稳定平均速度和最长稳定运行距离，包括在A1上提高16.1%的速度，在Cheetah上提高约2倍，在B1上提高约5倍。我们还展示了MorFiC在不同形态上减少价值预测误差方差并稳定优势估计，证明了改进的价值函数校准对应于更强的迁移性能。最后，我们展示了在两个Unitree Go1和Go2机器人上无需微调的零样本部署，表明评论者侧的调制是跨形态泛化的一种实用方法。
英文摘要：
Generalizing learned locomotion policies across quadrupedal robots with different morphologies remains a challenge. Policies trained on a single robot often fail when deployed on embodiments with different mass distributions, kinematics, joint limits, or actuation constraints, forcing per-robot retraining. Prior works have approached this primarily either by scaling via training across real or generated embodiments or using large architectures producing transferable policies which are both storage and compute heavy. We argue that both of these approaches circumvent a key failure mode in actor-critic learning: a shared value function tends to average incompatible value targets across embodiments, yielding miscalibrated advantages and fixing that helps transfer and reduce the compute cost. We present MorFiC, a reinforcement learning approach for zero-shot cross-morphology locomotion which fixes this issue using multiplicative critic conditioning on a morphology latent. Trained with a single source robot with morphology randomization in simulation, MorFiC achieves zero-shot transfer to total seven robots and reaching forward velocity competitive or surpassing scaling-based and morphology-conditioned PPO baselines for examples 1.98 m/s on AlienGo, where as additive PPO baselines remain below 0.65 m/s. MorFiC achieves this using single training robot within 2 hours of training time compared to tens or hundreds of hours required for scaling baselines training on multiple embodiments. We diagnose critic calibration via three metric: Explained variance, advantage sign-flip rate and policy gradient cosine, confirming our critic conditioning as determinant for policy update direction. Finally, we demonstrate zero-shot deployment on Unitree Go1 and Go2 robots without fine-tuning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
