基于扩散模型的人形机器人策略神经权重建模与理解
Modeling and Understanding Neural Weights of Humanoid Policies with Diffusion
AI总结：
本研究采用结合线性自编码器与扩散Transformer的潜在扩散框架，对Unitree G1机器人控制网络的权重进行生成式建模，生成可执行多种动作的网络，并发现其权重空间中存在反映运动学先验的共性结构。
摘要：
机器人学习领域的最新进展推动了人形机器人控制技术的快速发展，使其在现实世界中能实现越来越敏捷的行为。然而，人形机器人控制网络通常是通过传统训练流程得到的，而针对其学习到的权重分布进行直接建模、并在权重空间中生成功能性网络的研究相对较少。本研究探讨是否可通过高性能人形机器人控制网络的权重直接对其进行生成式建模。我们采用了一种结合线性自编码器与扩散Transformer的潜在扩散框架：自编码器将演员网络参数压缩至紧凑的潜在空间，扩散Transformer则对这些潜在变量的分布进行建模。我们在从Unitree G1机器人的强化学习过程中收集的网络上训练该框架，所提出的方法可生成可部署的网络，在现实世界中执行多种动作，包括行走、跳舞和旋转踢。除生成任务外，我们还进一步研究了学习到的权重空间，以更好地理解这些人形机器人控制网络所捕获的结构。我们在第一层权重中识别出一种结构原理，其始终反映机器人的运动学先验。值得注意的是，该运动学先验在强化学习（RL）和模仿学习（IL）两种训练方式下均会出现，表明它并非特定于某一优化过程，而可能反映了人形机器人实体相关的共性知识。综上，这些结果证明了在人形机器人网络权重空间中进行生成式建模的可行性，同时为其学习到的权重所编码的信息提供了初步见解。更多结果可在我们的项目网站获取：this https URL
英文摘要：
Recent advances in robot learning have driven rapid progress in humanoid control, enabling increasingly agile behaviors in the real world. However, humanoid control networks are typically obtained through conventional training procedures, while comparatively little attention has been given to directly modeling the distribution of their learned weights and generating functional networks in weight space. In this work, we investigate whether high-performing humanoid control networks can be generatively modeled directly through their weights. We adopt a latent diffusion framework that couples a linear autoencoder with a Diffusion Transformer. The autoencoder compresses actor network parameters into a compact latent space, and the Diffusion Transformer models the distribution of these latents. Trained on networks collected from reinforcement learning for the Unitree G1 robot, our approach generates deployable networks that perform various motions in the real world, including walking, dancing, and spin kick. Beyond generation, we further study the learned weight space to better understand the structure captured by these humanoid control networks. We identify a structural principle in the first-layer weights that consistently reflects the robot's kinematic prior. Notably, this kinematic prior emerges under both Reinforcement Learning (RL) and Imitation Learning (IL) training, suggesting that it is not specific to a particular optimization process but may instead reflect common knowledge associated with humanoid embodiment. Together, these results demonstrate the feasibility of generative modeling in humanoid network weight space, while providing initial insights into the information encoded by their learned weights. Additional results are available on our project website: https://humanoid-weight-diffusion.github.io/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
