EDAR：学习用于机器人操作的环境相关动作表示

中文摘要 AI 辅助

学习有效的动作表示对于机器人操作至关重要，原始控制轨迹往往存在噪声、冗余且难以直接建模。现有方法主要对动作流结构进行编码，将动作在环境中的作用视为隐含的。然而操作是为了改变世界，相同动作片段在不同场景下会产生不同结果，动作语义本质上依赖于环境。我们提出EDAR，一种将动作令牌基于可执行控制结构和预期视觉后果的环境相关动作表示。通过将电机命令与其环境条件效果相结合，EDAR促使学习到的动作空间捕捉交互语义而非仅仅是命令级模式。在模拟和真实机器人操作基准上的实验表明，EDAR改善了下游策略学习，尤其是在长期操作中。这些结果凸显了将动作表示基于可执行控制结构和环境条件视觉变化的重要性。

英文摘要

Learning effective action representations is critical for robotic manipulation, where raw control trajectories are often noisy, redundant, and difficult to model directly. Existing methods mainly encode the structure of the action stream itself, treating the role of actions in the environment as implicit. Yet manipulation is about changing the world: the same action segment can induce different outcomes under different scene contexts, making action semantics inherently environment-dependent. We propose EDAR, an Environment-Dependent Action Representation that grounds action tokens in both executable control structure and expected visual consequences. By coupling motor commands with their environment-conditioned effects, EDAR encourages the learned action space to capture interaction semantics rather than merely command-level patterns. Experiments on simulated and real-robot manipulation benchmarks demonstrate that EDAR improves downstream policy learning, especially in long-horizon manipulation. These results highlight the importance of grounding action representations in executable control structure and environment-conditioned visual change.

发表机构

• College of Intelligent Robotics and Advanced Manufacturing, Fudan University（复旦大学智能机器人与先进制造学院）
• Shanghai Key Lab of Intelligent Information Processing, College of Computer Science and Artificial Intelligence, Fudan University（复旦大学计算机科学与人工智能学院智能信息处理上海市重点实验室）
• TeleAI, China Telecom（中国电信天翼人工智能公司）

机构由 AI 辅助整理，请以论文原文为准。