DexFuture: 用于双手灵巧工具使用的分层未来状态视觉运动目标
DexFuture: Hierarchical Future-State Visuomotor Targeting for Bimanual Dexterous Tool Use
AI总结：
提出DexFuture分层系统，通过高层未来状态视觉运动目标预测器和低层目标条件结构化灵巧策略，实现双手灵巧工具使用，达到90%的特权oracle性能，运行速度60Hz，比DexWM式CEM规划快约250倍。
摘要：
双手灵巧工具使用对机器人来说仍然具有挑战性，因为手部配置维度高，且手-工具-物体动力学和接触复杂。大多数现有控制策略依赖于演示提供的未来配置参考，而未来动作条件世界模型需要对高维动作序列进行缓慢的在线规划。一个重大挑战是生成动态一致的未来参考轨迹，而不依赖于演示中的特权状态或缓慢的反事实规划。我们提出DexFuture，一个分层系统，将高层未来状态视觉运动目标预测器与低层目标条件结构化灵巧策略耦合。基于自我中心RGB、本体感觉和几何历史，高层预测器构建结构化的手-工具-物体视觉运动嵌入，并使用水平条件Transformer生成多步未来目标轨迹。然后，低层策略通过目标条件每链接Transformer跟踪这些轨迹。这种分层结构将粗略的未来参考生成与细粒度的动作控制解耦，并将缓慢的长时域语义预测与高频执行解耦。在OakInk2双手工具使用任务上，DexFuture达到了90%的特权oracle性能，而无参考策略仅为7%。DexFuture以60Hz运行，比DexWM风格的交叉熵方法（CEM）规划（使用未来动作条件世界模型）快约250倍。
英文摘要：
Bimanual dexterous tool use remains challenging for robots due to high-dimensional hand configurations and complex hand-tool-object dynamics and contact. Most existing control policies depend on future configuration references provided from demonstrations, while future action-conditioned world models require slow online planning over high-dimensional action sequences. A significant challenge is generating a dynamically consistent future reference trajectory without relying on privileged states from demonstrations or slow counterfactual planning. We propose DexFuture, a hierarchical system that couples a high-level Future-State Visuomotor Target Predictor with a low-level Target-Conditioned Structured Dexterous Policy. Conditioned on egocentric RGB, proprioceptive and geometric history, the high-level predictor constructs structured hand-tool-object visuomotor embeddings and uses a horizon-conditioned transformer to generate a multi-step future target trajectory. Then, the low-level policy tracks them with a target-conditioned per-link transformer. This hierarchy decouples coarse future reference generation from fine-grained action control, and slow long-horizon semantic prediction from high-frequency execution. On OakInk2 bimanual tool-use tasks, DexFuture achieves 90% of the privileged-oracle performance, compared to 7% for a no-reference policy. DexFuture operates at 60 Hz, approximately 250 times faster than DexWM-style Cross-Entropy Method (CEM) planning with a future action-conditioned world model.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
