TacUMI: A Multi-Modal Universal Manipulation Interface for Contact-Rich Tasks
School of Computation, Information and Technology, Technical University of Munich(技术大学慕尼黑计算、信息与技术学院);Agile Robots SE(敏捷机器人公司);State Key Laboratory for Novel Software Technology and the School of Science and Technology, Nanjing University (Suzhou Campus)(南京大学软件新技术国家重点实验室及科学与技术学院(苏州校区));Shanghai University(上海大学)
Task decomposition is critical for understanding and learning complex long-horizon manipulation tasks. Especially for tasks involving rich physical interactions, relying solely on visual observations and robot proprioceptive information often fails to reveal the underlying event transitions. This raises the requirement for efficient collection of high-quality multi-modal data as well as robust segmentation method to decompose demonstrations into meaningful modules. Building on the idea of the handheld demonstration device Universal Manipulation Interface (UMI), we introduce TacUMI, a multi-modal data collection system that integrates additionally ViTac sensors, force-torque sensor, and pose tracker into a compact, robot-compatible gripper design, which enables synchronized acquisition of all these modalities during human demonstrations. We then propose a multi-modal segmentation framework that leverages temporal models to detect semantically meaningful event boundaries in sequential manipulations. Evaluation on a challenging cable mounting task shows more than 90 percent segmentation accuracy and highlights a remarkable improvement with more modalities, which validates that TacUMI establishes a practical foundation for both scalable collection and segmentation of multi-modal demonstrations in contact-rich tasks.
TouchGuide: Inference-Time Steering of Visuomotor Policies via Touch Guidance
Shanghai Jiao Tong University(上海交通大学);Xense Robotics(Xense机器人公司);Sun Yat-sen University(中山大学);Oxford(牛津大学);Shanghai AI Laboratory(上海人工智能实验室);University of Science and Technology of China(中国科学技术大学);UCSD(加州大学圣地亚哥分校)
TACT: Humanoid Whole-body Contact Manipulation through Deep Imitation Learning with Tactile Modality
CNRS-AIST JRL (Joint Robotics Laboratory), IRL and National Institute of Advanced Industrial Science and Technology (AIST)(CNRS-AIST联合机器人实验室、IRL和国家先进工业科学与技术研究院);Tokyo University of Science(东京科学大学);University of Tsukuba(茨口大学)
Cross-Modal Visuo-Tactile Representation Learning with Action Chunking Transformers for Contact-Rich Manipulation
Guangdong Institute of Intelligence Science and Technology(广东智能科学与技术研究院);Innovation and Research and Development Department, BoardWare Information System Company Ltd.(创新与研发部,BoardWare信息系统有限公司);School of Artificial Intelligence, Nanjing Agricultural University(人工智能学院,南京农业大学);School of Computing, National University of Singapore(计算机学院,新加坡国立大学);School of Electronic Engineering and Automation, Guilin University of Electronic Technology(电子工程与自动化学院,桂林电子科技大学)
Multi-Modal Manipulation via Multi-Modal Policy Consensus
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Columbia University(哥伦比亚大学);Massachusetts Institute of Technology(麻省理工学院);Harvard University(哈佛大学)
LEMMo-Plan: LLM-Enhanced Learning from Multi-Modal Demonstration for Planning Sequential Contact-Rich Manipulation Tasks
School of Computation, Information and Technology, Technical University of Munich(慕尼黑工业大学计算、信息与技术学院);Aalto University(阿尔托大学);Nanjing University (Suzhou Campus)(南京大学苏州校区);Mohamed Bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
Spacetime Optimal-Transport Attention for Visuo-Haptic Imitation Learning of Contact-Rich Manipulation
Robotics Research Centre, School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore(机器人研究中心,机械与航空航天工程学院,南洋理工大学,新加坡);Wings Robotics(Wings机器人)
ManipForce: Force-Guided Policy Learning with Frequency-Aware Representation for Contact-Rich Manipulation
Department of AI Convergence, Gwangju Institute of Science and Technology (GIST)(人工智能融合系,全州科学技术院);Department of AI Machinery, Korea Institute of Machinery & Materials (KIMM)(人工智能机械系,韩国机械材料研究院)
OWMM-Agent: Open World Mobile Manipulation With Multi-modal Agentic Data Synthesis
Shanghai AI Laboratory(上海人工智能实验室);School of Computing, National University of Singapore(新加坡国立大学计算机学院);Shanghai Jiaotong University(上海交通大学);Tsinghua University(清华大学)
Mitigating Cross-Modal Distraction and Ensuring Geometric Feasibility via Affordance-Guided and Self-Consistent MLLMs for Task Planning in Instruction-Following Manipulation
Department of Computer Science, National Yang Ming Chiao Tung University(计算机科学系,阳明交通大学);Department of Computer Science and Information Engineering, National Taiwan University(计算机科学与信息工程系,台湾大学)
Towards Dexterous Embodied Manipulation via Deep Multi-Sensory Fusion and Sparse Expert Scaling
College of Intelligent Robotics and Advanced Manufacturing, Fudan University(智能机器人与先进制造学院,复旦大学);School of Information Science and Engineering, Harbin Institute of Technology(信息科学与工程学院,哈尔滨工业大学)
VTLA: Vision-Tactile-Language-Action Model with Preference Learning for Insertion Manipulation
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(多模态人工智能系统国家重点实验室,自动化研究所,中国科学院);Samsung R&D Institute China–Beijing(三星中国北京研发中心);Beijing Academy of Artificial Intelligence(北京人工智能研究院)