Soft object manipulation has recently gained popularity within the robotics community due to its potential applications in many economically important areas. Although great progress has been recently achieved in these types of tasks, most state-of-the-art methods are case-specific; They can only be used to perform a single deformation task (e.g. bending), as their shape representation algorithms typically rely on "hard-coded" features. In this paper, we present LaSeSOM, a new feedback latent representation framework for semantic soft object manipulation. Our new method introduces internal latent representation layers between low-level geometric feature extraction and high-level semantic shape analysis; This allows the identification of each compressed semantic function and the formation of a valid shape classifier from different feature extraction levels. The proposed latent framework makes soft object representation more generic (independent from the object's geometry and its mechanical properties) and scalable (it can work with 1D/2D/3D tasks). Its high-level semantic layer enables to perform (quasi) shape planning tasks with soft objects, a valuable and underexplored capability in many soft manipulation tasks. To validate this new methodology, we report a detailed experimental study with robotic manipulators.
DeformNet: Latent Space Modeling and Dynamics Prediction for Deformable Object Manipulation
Tsinghua-Berkeley Shenzhen Institute(清华-伯克利深圳学院);Tsinghua Shenzhen International Graduate School(清华大学深圳国际研究生院);Tsinghua University(清华大学);RISC-V International Open Source Laboratory(RISC-V国际开源实验室);Institute for Interdisciplinary Information Sciences (IIIS)(交叉信息研究院)
EmbodiedVAE: Disentangled Video VAE for Efficient and Controllable Embodied Manipulation
Beihang University(北京航空航天大学);Zhongguancun Academy(中关村学院);University of Science and Technology of China(中国科学技术大学);National University of Singapore(新加坡国立大学);CASIA, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所(CASIA));Microsoft Research Asia(微软亚洲研究院)
Learning Action Manifold with Multi-view Latent Priors for Robotic Manipulation
Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education, China(机器智能与先进计算国家重点实验室,教育部,中国);AMap, Alibaba Group(阿里的AMap);Xi’an Jiaotong University(西安交通大学);Shenzhen University of Advanced Technology(深圳先进技术大学)
HiMem-WAM: Hierarchical Memory-Gated World Action Models for Robotic Manipulation
The University of Hong Kong(香港大学);INFIFORCE;Huazhong University of Science and Technology(华中科技大学);Tsinghua University(清华大学);Wuhan University(武汉大学);Southern University of Science and Technology(南方科技大学)
CoLA-Flow Policy: Temporally Coherent Imitation Learning via Continuous Latent Action Flow Matching for Robotic Manipulation
State Key Laboratory of Robotics and System, Harbin Institute of Technology(机器人系统国家重点实验室,哈尔滨工业大学);The University of Sydney(悉尼大学);Honor Device Co., Ltd.(荣耀设备有限公司)
Latent Conditioned Loco-Manipulation Using Motion Priors
LAAS-CNRS, Université de Toulouse(CNRS拉法耶特研究所、图卢兹大学);Artificial and Natural Intelligence Toulouse Institute(图卢兹人工智能与自然智能研究所);University of Edinburgh, School of Informatics(爱丁堡大学信息学院)
ConLA: Contrastive Latent Action Learning from Human Videos for Robotic Manipulation
Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));State Key Laboratory of Mobile Network and Mobile Multimedia Technology, Shenzhen(移动网络与移动多媒体技术国家重点实验室(深圳));ZTE Corporation(中兴通讯);Shanghai Jiao Tong University(上海交通大学);Central South University(中南大学)
DeltaWorld: Physically Consistent Interactive World Simulators via Action-Conditioned Latent Increment Learning
Imprintx Robotics(印迹机器人);Electric Power Research Institute of State Grid Jibei Electric Power Co., Ltd.(国网冀北电力有限公司电力科学研究院);North China Electric Power Research Institute Co., Ltd.(华北电力科学研究院有限责任公司);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)
LaST-HD: Learning Latent Physical Reasoning from Scalable Human Data for Robot Manipulation
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University(北京大学计算机学院多媒体信息处理国家重点实验室);The Chinese University of Hong Kong(香港中文大学);Simplexity Robotics;Aether Tech
From Imagined Futures to Executable Actions: Mixture of Latent Actions for Robot Manipulation
School of Data Science, Fudan University(复旦大学数据科学学院);Shanghai Innovation Institute(上海创新研究院);Imperial College London(伦敦帝国理工学院);University of Surrey(萨里大学)