结构动作变换器用于3D灵巧操作
Structural Action Transformer for 3D Dexterous Manipulation
AI总结：
本文提出结构动作变换器，通过结构化视角解决高自由度机械手的跨身体技能转移问题，实现更高效的灵巧操作。
摘要：
通过从异构数据集进行模仿学习实现机器人的人类水平灵巧性受到跨身体技能转移的挑战，特别是对于高自由度机械手。现有方法通常依赖于2D观察和时间中心的动作表示，难以捕捉3D空间关系，并且无法处理身体异质性。本文提出了结构动作变换器（SAT），一种新的3D灵巧操作策略，通过引入以结构为中心的视角来挑战这一范式。我们将每个动作片段不视为时间序列，而是视为变量长度、无序的关节轨迹序列。这种结构化形式使变换器能够原生处理异质身体，将关节数量视为可变序列长度。为编码结构先验并解决歧义，我们引入了具身关节代码本，嵌入每个关节的功能角色和运动学属性。我们的模型通过连续时间流匹配目标从3D点云学习生成这些轨迹。我们通过在大规模异构数据集上预训练并在模拟和现实世界灵巧操作任务上微调来验证我们的方法。我们的方法在所有基线中表现一致，证明了其优越的样本效率和有效的跨身体技能转移。这种以结构为中心的表示为扩展高自由度、异质操纵器的策略提供了新路径。
英文摘要：
Achieving human-level dexterity in robots via imitation learning from heterogeneous datasets is hindered by the challenge of cross-embodiment skill transfer, particularly for high-DoF robotic hands. Existing methods, often relying on 2D observations and temporal-centric action representation, struggle to capture 3D spatial relations and fail to handle embodiment heterogeneity. This paper proposes the Structural Action Transformer (SAT), a new 3D dexterous manipulation policy that challenges this paradigm by introducing a structural-centric perspective. We reframe each action chunk not as a temporal sequence, but as a variable-length, unordered sequence of joint-wise trajectories. This structural formulation allows a Transformer to natively handle heterogeneous embodiments, treating the joint count as a variable sequence length. To encode structural priors and resolve ambiguity, we introduce an Embodied Joint Codebook that embeds each joint's functional role and kinematic properties. Our model learns to generate these trajectories from 3D point clouds via a continuous-time flow matching objective. We validate our approach by pre-training on large-scale heterogeneous datasets and fine-tuning on simulation and real-world dexterous manipulation tasks. Our method consistently outperforms all baselines, demonstrating superior sample efficiency and effective cross-embodiment skill transfer. This structural-centric representation offers a new path toward scaling policies for high-DoF, heterogeneous manipulators.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
