Joint Discovery of Object States and Manipulation Actions
Département d’informatique de l’ENS, École normale supérieure, CNRS, PSL Research University（巴黎高等师范学院计算机科学系、法国国家科学研究中心、巴黎文理研究大学）；INRIA（法国国家信息与自动化研究所）；Czech Institute of Informatics, Robotics and Cybernetics at the Czech Technical University in Prague（布拉格捷克理工大学信息学、机器人与控制论研究所）；Department of CS & OR (DIRO), Université de Montréal（蒙特利尔大学计算机科学与运筹学系）
Jean-Baptiste Alayrac, Josev Sivic, Ivan Laptev, Simon Lacoste-Julien
Many human activities involve object manipulations aiming to modify the object state. Examples of common state changes include full/empty bottle, open/closed door, and attached/detached car wheel. In this work, we seek to automatically discover the states of objects and the associated manipulation actions. Given a set of videos for a particular task, we propose a joint model that learns to identify object states and to localize state-modifying actions. Our model is formulated as a discriminative clustering cost with constraints. We assume a consistent temporal order for the changes in object states and manipulation actions, and introduce new optimization techniques to learn model parameters without additional supervision. We demonstrate successful discovery of seven manipulation actions and corresponding object states on a new dataset of videos depicting real-life object manipulations. We show that our joint formulation results in an improvement of object state discovery by action recognition and vice versa.
Structural Action Transformer for 3D Dexterous Manipulation
MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China（脑启发智能感知与认知国家重点实验室，中国科学技术大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（人工智能研究院，合肥综合性国家科学中心）
AffordanceWAM: Affordance-Aware Joint World-Action Modeling for Robot Manipulation
The University of Hong Kong（香港大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Peking University（北京大学）；The Chinese University of Hong Kong（香港中文大学）；National University of Singapore（新加坡国立大学）；Knowin AI（考拉悠然科技）
CLIP-PAE: Projection-Augmentation Embedding to Extract Relevant Features for a Disentangled, Interpretable, and Controllable Text-Guided Face Manipulation
TONAV: Task-Oriented Navigation and Action-Velocity Chunk Learning for Articulated Object Quadrupedal Mobile Manipulation
School of Artificial Intelligence and Robotics, Hunan University（湖南大学人工智能与机器人学院）；National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University（湖南大学机器人视觉感知与控制技术国家工程研究中心）；College of Semiconductors (College of Integrated Circuits), Hunan University（湖南大学半导体学院（集成电路学院））
Novel Class Discovery for Point Cloud Segmentation via Joint Learning of Causal Representation and Reasoning
School of Artificial Intelligence, College of Intelligence and Computing, Tianjin University, China（人工智能学院、智能与计算学院、天津大学）；School of Computer Science and Information Engineering, Hefei University of Technology, China（计算机科学与信息工程学院、合肥工业大学）
SaPaVe: Towards Active Perception and Manipulation in Vision-Language-Action Models for Robotics
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（信息多媒体国家重点实验室，计算机科学学院，北京大学）；School of Software, Beihang University（软件学院，北航）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）
