使用Buckingham Pi定理在强化学习中进行零样本策略迁移
Zero-Shot Policy Transfer in Reinforcement Learning using Buckingham's Pi Theorem
AI总结：
本文提出基于Buckingham Pi定理的零样本迁移方法，通过无量纲空间缩放策略输入输出，无需重训即可提升RL策略在新物理参数环境中的泛化能力与鲁棒性。
摘要：
强化学习（RL）策略通常难以泛化到具有不同物理参数的新机器人、任务或环境，这一挑战限制了其在现实世界中的适用性。本文提出了一种基于Buckingham Pi定理的简单零样本迁移方法来解决这一局限性。该方法通过无量纲空间缩放预训练策略的输入（观测）和输出（动作），使其适应新的系统环境，且无需重新训练。该方法在复杂度递增的三个环境中与朴素迁移基线进行了对比评估：一个模拟摆、一个用于sim-to-real验证的物理摆，以及高维的HalfCheetah。结果表明，在动态相似的上下文中，缩放迁移没有性能损失。此外，在非相似上下文中，缩放策略始终优于朴素迁移，显著扩大了原始策略保持有效的上下文范围。这些发现表明，量纲分析为增强RL策略的鲁棒性和泛化能力提供了强大且实用的工具。
英文摘要：
Reinforcement learning (RL) policies often fail to generalize to new robots, tasks, or environments with different physical parameters, a challenge that limits their real-world applicability. This paper presents a simple, zero-shot transfer method based on Buckingham's Pi Theorem to address this limitation. The method adapts a pre-trained policy to new system contexts by scaling its inputs (observations) and outputs (actions) through a dimensionless space, requiring no retraining. The approach is evaluated against a naive transfer baseline across three environments of increasing complexity: a simulated pendulum, a physical pendulum for sim-to-real validation, and the high-dimensional HalfCheetah. Results demonstrate that the scaled transfer exhibits no loss of performance on dynamically similar contexts. Furthermore, on non-similar contexts, the scaled policy consistently outperforms the naive transfer, significantly expanding the volume of contexts where the original policy remains effective. These findings demonstrate that dimensional analysis provides a powerful and practical tool to enhance the robustness and generalization of RL policies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
