Towards Dexterous Embodied Manipulation via Deep Multi-Sensory Fusion and Sparse Expert Scaling
College of Intelligent Robotics and Advanced Manufacturing, Fudan University（智能机器人与先进制造学院，复旦大学）；School of Information Science and Engineering, Harbin Institute of Technology（信息科学与工程学院，哈尔滨工业大学）
Yirui Sun, Guangyu Zhuge, Keliang Liu, Jie Gu, Zhihao xia, Qionglin Ren, Chunxu tian, Zhongxue Ga
Realizing dexterous embodied manipulation necessitates the deep integration of heterogeneous multimodal sensory inputs. However, current vision-centric paradigms often overlook the critical force and geometric feedback essential for complex tasks. This paper presents DeMUSE, a Deep Multimodal Unified Sparse Experts framework leveraging a Diffusion Transformer to integrate RGB, depth, and 6-axis force into a unified serialized stream. Adaptive Modality-specific Normalization (AdaMN) is employed to recalibrate modality-aware features, mitigating representation imbalance and harmonizing the heterogeneous distributions of multi-sensory signals. To facilitate efficient scaling, the architecture utilizes a Sparse Mixture-of-Experts (MoE) with shared experts, increasing model capacity for physical priors while maintaining the low inference latency required for real-time control. A Joint denoising objective synchronously synthesizes environmental evolution and action sequences to ensure physical consistency. Achieving success rates of 83.2% and 72.5% in simulation and real-world trials, DeMUSE demonstrates state-of-the-art performance, validating the necessity of deep multi-sensory integration for complex physical interactions.
Grasp What You Want: Embodied Dexterous Grasping System Driven by Your Voice
Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳市人工智能与机器人研究院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；South China University of Technology（华南理工大学）
MIPD: A Multi-sensory Interactive Perception Dataset for Embodied Intelligent Driving
College of Information Science and Technology, Beijing University of Chemical Technology（北京化工大学信息科学与技术学院）；Institute of Ophthalmology, Beijing Tongren Hospital（北京同仁医院眼科研究所）；College of Information Science and Engineering, Yanshan University（燕山大学信息科学与工程学院）；College of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；College of Mechanics and Vehicles, Beijing Institute of Technology（北京理工大学机械与车辆学院）；College of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术学院）
Towards Robotic Dexterous Hand Intelligence: A Survey
University of Liverpool（利物浦大学）；Xi’an Jiaotong-Liverpool University（西安交通大学-利物浦大学）；Duke Kunshan University（杜克-昆山大学）；The Chinese University of Hong Kong（香港中文大学）
Towards Human-Like Manipulation through RL-Augmented Teleoperation and Mixture-of-Dexterous-Experts VLA
Shanghai Jiao Tong University（上海交通大学）；Sharpa；Shanghai AI Lab（上海人工智能实验室）；Zhongguancun Academy（中关村学院）；National University of Singapore（新加坡国立大学）；Shanghai Innovation Institute（上海创新研究院）
Adaptive Visuo-Tactile Fusion with Predictive Force Attention for Dexterous Manipulation
Center on Frontiers of Computing Studies, School of Computer Science, Peking University（前沿计算研究中心，计算机科学学院，北京大学）；PKU-Agibot Lab, School of Computer Science, Peking University（PKU-Agibot实验室，计算机科学学院，北京大学）；National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University（多媒体信息处理国家重点实验室，计算机科学学院，北京大学）；Ant Group（蚂蚁集团）；Institute for Artificial Intelligence, Peking University（人工智能研究院，北京大学）
Suction Leap-Hand: Suction Cups on a Multi-fingered Hand Enable Embodied Dexterity and In-Hand Teleoperation
School of Informatics, University of Edinburgh（信息学院，爱丁堡大学）；School of Physics, Engineering and Technology, University of York（物理、工程与技术学院，约克大学）；Tsinghua University（清华大学）
ObjRetarget: An Object-Aware Motion Retargeting Framework with Anthropomorphic Arm Constraints and Polyhedral Hand Modeling
School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）人工智能学院）；School of Computing, University of Portsmouth（朴茨茅斯大学计算学院）
Structural Action Transformer for 3D Dexterous Manipulation
MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China（脑启发智能感知与认知国家重点实验室，中国科学技术大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（人工智能研究院，合肥综合性国家科学中心）
NoContactNoWorries: Estimating Contact through Vision and Proprioception for In-Hand Dexterous Manipulation
Robotics Research Center (RRC), International Institute of Information Technology (IIIT), Hyderabad, India（印度海得拉巴国际信息技术研究所机器人研究中心）；Department of Computer Science, The University of Manchester（曼彻斯特大学计算机科学系）
A Survey: Learning Embodied Intelligence from Physical Simulators and World Models
Nanjing University（南京大学）；University of Hong Kong（香港大学）；Central South University（中南大学）；Horizon Robotics（Horizon机器人）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Shanghai Jiao Tong University（上海交通大学）；Technical University of Munich（慕尼黑技术大学）；Tsinghua University（清华大学）
EmbodiedSmith: Scaling Embodied Data through Recursive Self-Improvement Flywheel in Simulation
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Xiaomi Robotics（小米机器人）；Southeastern University（东南大学）；Tsinghua University（清华大学）；Zhejiang University（浙江大学）；Westlake University（西湖大学）；Symbiosis Robotics（共生机器人）；Peking University（北京大学）；Xi’an Jiaotong University（西安交通大学）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）
The Role of Touch: Towards Optimal Tactile Sensing Distribution in Anthropomorphic Hands for Dexterous In-Hand Manipulation
VisLab, Institute for Systems；Soft Robotics Lab National University of Singapore, Singapore；BRAIR Lab, BioRobotics Institute Scuola Superiore Sant'Anna, Pisa, Italia
ROMA: LLM System for Real-World Object-Centric Multi-Sensory Active Perception
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Beijing Key Laboratory of Research on Large Models and Intelligent Governance（北京大模型与智能治理重点实验室）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；Beijing Jiaotong University（北京交通大学）；State Key Laboratory of Multimedia Information Processing, Peking University（北京大学多媒体信息处理国家重点实验室）；AresoX
ManipulationNet: An Infrastructure for Benchmarking Real-World Robot Manipulation with Physical Skill Challenges and Embodied Multimodal Reasoning
Rice University（里士大学）；U.S. National Institute of Standards and Technology（美国国家标准与技术研究院）；Massachusetts Institute of Technology（麻省理工学院）；Karlsruhe Institute of Technology（卡尔斯鲁厄技术大学）；Autodesk Research（Autodesk研究）；University of California, Berkeley（加州大学伯克利分校）；KTH Royal Institute of Technology（皇家理工学院）；Tsinghua University（清华大学）；Columbia University（哥伦比亚大学）；ASTM International（美国材料与试验协会）；Carnegie Mellon University（卡内基梅隆大学）；German Aerospace Center (DLR)（德国航空航天中心）；University of Texas at Dallas（德克萨斯大学达拉斯分校）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；NVIDIA Research（NVIDIA研究）
Humanoid Agent via Embodied Chain-of-Action Reasoning with Multimodal Foundation Models for Zero-Shot Loco-Manipulation
Embodied AI and Robotics (AIR) Lab, New York University, New York, USA and NYUAD Center for Artificial Intelligence and Robotics, New York University Abu Dhabi, Abu Dhabi, UAE（纽约大学Embodied AI和机器人实验室及纽约大学阿布扎赫尔人工智能与机器人中心）；Harvard AI and Robotics Lab, Harvard University, Boston, USA（哈佛大学人工智能与机器人实验室）；Department of Computer Science, University College London, London, UK（伦敦大学学院计算机科学系）；Department of Computer Science, University of Liverpool, UK（利物浦大学计算机科学系）；NYUAD Center for Artificial Intelligence and Robotics, New York University Abu Dhabi, Abu Dhabi, UAE（纽约大学阿布扎赫尔人工智能与机器人中心）
Robot Synesthesia: In-Hand Manipulation with Visuotactile Sensing
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of California Berkeley（加州大学伯克利分校）；Dongguk University（东国大学）；Tsinghua University（清华大学）；University of California San Diego（加州大学圣迭戈分校）
