通过强化学习增强的遥控操作与混合的灵巧专家VLA实现人机操作
Towards Human-Like Manipulation through RL-Augmented Teleoperation and Mixture-of-Dexterous-Experts VLA
AI总结：
本文提出IMCopilot和MoDE-VLA框架，通过强化学习和多模态融合提升机器人灵巧操作能力，实现接触丰富的手内任务性能提升。
摘要：
尽管视觉-语言-动作（VLA）模型在机器人操作中表现出色，但其应用大多局限于低自由度末端执行器进行简单的视觉引导的拾取和放置任务。将这些模型扩展到人似、双臂灵巧操作——特别是接触丰富的手内操作——引入了高保真数据采集、多技能学习和多模态感觉融合的关键挑战。在本文中，我们提出了一种综合框架，以解决这些瓶颈，基于两个组件。首先，我们引入IMCopilot（手内操作助手），一套通过强化学习训练的原子技能，扮演双重角色：它作为共享自主助手，简化遥控数据采集，并作为可调用的低层执行原语用于VLA。其次，我们提出MoDE-VLA（混合灵巧专家VLA），一种将异构力和触觉模态无缝整合到预训练VLA骨干中的架构。通过利用残差注入机制，MoDE-VLA在不降低模型预训练知识的情况下实现接触感知细化。我们验证了我们的方法在四个渐进复杂任务上的有效性，在灵巧接触丰富的任务中，成功率相比基线提高了两倍。
英文摘要：
While Vision-Language-Action (VLA) models have demonstrated remarkable success in robotic manipulation, their application has largely been confined to low-degree-of-freedom end-effectors performing simple, vision-guided pick-and-place tasks. Extending these models to human-like, bimanual dexterous manipulation-specifically contact-rich in-hand operations-introduces critical challenges in high-fidelity data acquisition, multi-skill learning, and multimodal sensory fusion. In this paper, we propose an integrated framework to address these bottlenecks, built upon two components. First, we introduce IMCopilot (In-hand Manipulation Copilot), a suite of reinforcement learning-trained atomic skills that plays a dual role: it acts as a shared-autonomy assistant to simplify teleoperation data collection, and it serves as a callable low-level execution primitive for the VLA. Second, we present MoDE-VLA (Mixture-of-Dexterous-Experts VLA), an architecture that seamlessly integrates heterogeneous force and tactile modalities into a pretrained VLA backbone. By utilizing a residual injection mechanism, MoDE-VLA enables contact-aware refinement without degrading the model's pretrained knowledge. We validate our approach on four tasks of escalating complexity, demonstrating doubled success rate improvement over the baseline in dexterous contact-rich tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
