arXiv 2602.055132026-08-18cs.ROcs.AI

DECO：解耦多模态扩散变压器用于配备插件触觉适配器的双臂灵巧操作

DECO: Decoupled Multimodal Diffusion Transformer for Bimanual Dexterous Manipulation with a Plugin Tactile Adapter

Beijing Academy of Artificial Intelligence, Beijing, China（北京人工智能研究院）；School of Computation, Information and Technology, Technical University of Munich, Garching, Germany（慕尼黑技术大学计算与信息学院）；Department of Shenyang Institute of Computing Technology, University of Chinese Academy of Sciences, Beijing, China（中国科学院沈阳计算技术研究所部门）；State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China（新型软件技术国家重点实验室）；Department of Computer Science and Technology, Tsinghua University, Beijing, China（清华大学计算机科学与技术系）

Xukun Li, Yu Sun, Lei Zhang, Bosheng Huang, Yibo Peng, Yuan Meng, Haojun Jiang, Shaoxuan Xie, Guocai Yao, Alois Knoll, Zhenshan Bing, Xinlong Wang, Zhenguo Sun

备注：: 25 pages, 8 figures. Project Page: https://baai-humanoid.github.io/DECO-webpage/

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

DECO通过解耦多模态输入和触觉适配器，实现了双臂灵巧操作的高效整合与高成功率

摘要：

双臂灵巧操作依赖于整合多模态输入以执行复杂的现实任务。为了解决有效结合这些模态的挑战，我们提出了DECO，一种解耦的多模态扩散变压器，通过专门的条件路径解耦视觉、本体感觉和触觉信号，实现多模态输入的结构化和可控整合，并配备轻量级适配器以高效注入额外信号。此外，我们发布了DECO-50数据集，用于配备触觉传感的双臂灵巧操作，包含50小时的数据和超过500万帧，通过远程操作在真实的双臂机器人上收集。我们训练DECO在DECO-50上，并进行了超过2000次机器人运行的广泛现实评估。实验结果表明，DECO在所有任务上均取得最佳性能，平均成功率为72.25%，比基线提高了21%。此外，触觉适配器在所有任务上带来了额外的10.25%平均成功率，并在复杂接触丰富的任务上获得20%的提升，同时仅调整了模型参数的不到10%。

英文摘要：

Bimanual dexterous manipulation relies on integrating multimodal inputs to perform complex real-world tasks. To address the challenges of effectively combining these modalities, we propose DECO, a decoupled multimodal diffusion transformer that disentangles vision, proprioception, and tactile signals through specialized conditioning pathways, enabling structured and controllable integration of multimodal inputs, with a lightweight adapter for parameter-efficient injection of additional signals. Alongside DECO, we release DECO-50 dataset for bimanual dexterous manipulation with tactile sensing, consisting of 50 hours of data and over 5M frames, collected via teleoperation on real dual-arm robots. We train DECO on DECO-50 and conduct extensive real-world evaluation with over 2,000 robot rollouts. Experimental results show that DECO achieves the best performance across all tasks, with a 72.25% average success rate and a 21% improvement over the baseline. Moreover, the tactile adapter brings an additional 10.25% average success rate across all tasks and a 20% gain on complex contact-rich tasks while tuning less than 10% of the model parameters.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。