Contrary to the vast literature in modeling, perceiving, and understanding agent-object (e.g., human-object, hand-object, robot-object) interaction in computer vision and robotics, very few past works have studied the task of object-object interaction, which also plays an important role in robotic manipulation and planning tasks. There is a rich space of object-object interaction scenarios in our daily life, such as placing an object on a messy tabletop, fitting an object inside a drawer, pushing an object using a tool, etc. In this paper, we propose a unified affordance learning framework to learn object-object interaction for various tasks. By constructing four object-object interaction task environments using physical simulation (SAPIEN) and thousands of ShapeNet models with rich geometric diversity, we are able to conduct large-scale object-object affordance learning without the need for human annotations or demonstrations. At the core of technical contribution, we propose an object-kernel point convolution network to reason about detailed interaction between two objects. Experiments on large-scale synthetic data and real-world data prove the effectiveness of the proposed approach. Please refer to the project webpage for code, data, video, and more materials: https://cs.stanford.edu/~kaichun/o2oafford
RAGNet: Large-scale Reasoning-based Affordance Segmentation Benchmark towards General Grasping
The Chinese University of Hong Kong(香港中文大学);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);Dexmal;Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);SKL-IOTSC, CIS, University of Macau(澳门科学馆-物联网与智能系统研究中心,澳门大学)
AffordanceWAM: Affordance-Aware Joint World-Action Modeling for Robot Manipulation
The University of Hong Kong(香港大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Peking University(北京大学);The Chinese University of Hong Kong(香港中文大学);National University of Singapore(新加坡国立大学);Knowin AI(考拉悠然科技)
RoboPCA: Pose-centered Affordance Learning from Human Demonstrations for Robot Manipulation
Key Laboratory of AI Safety of CAS, Institute of Computing Technology, Chinese Academy of Sciences(中国科学院人工智能安全重点实验室、计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学)
Learning Granularity-Aware Affordances from Human-Object Interaction for Tool-Based Functional Dexterous Grasping
School of Artificial Intelligence and Robotics, Hunan University(人工智能与机器人学院,湖南大学);National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University(机器人视觉感知与控制技术国家工程研究中心,湖南大学)
RoboAfford++: A Generative AI-Enhanced Dataset for Multimodal Affordance Learning in Robotic Manipulation and Navigation
Xiaomi EV(小米电动车);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学 Gallagher人工智能学院);School of Computer Science and Information Engineering, Hefei University of Technology(合肥工业大学计算机科学与信息工程学院)
DORA: Object Affordance-Guided Reinforcement Learning for Dexterous Robotic Manipulation
TAMS (Technical Aspects of Multimodal Systems), Department of Informatics, Universität Hamburg(技术多模态系统(TAMS),信息学院,汉堡大学);Agile Robots AG(敏捷机器人公司);Technical University of Munich(慕尼黑技术大学)
Open-Vocabulary Affordance Detection using Knowledge Distillation and Text-Point Correlation
FPT Software AI Center(FPT软件人工智能中心);TU Wien(维也纳工业大学);Imperial College London(帝国理工学院);University of Arkansas(阿肯色大学);Ton Duc Thang University(孙德胜大学);University of Liverpool(利物浦大学)
3D Affordance Keypoint Detection for Robotic Manipulation
Advanced Robotics Centre, National University of Singapore(新加坡国立大学先进机器人中心);Department of Mechanical Engineering, National University of Singapore(新加坡国立大学机械工程系)
Affordance-R1: Reinforcement Learning for Generalizable Affordance Reasoning in Multimodal Large Language Model
The Hong Kong University of Science and Technology (GZ)(香港科技大学(广州));National University of Singapore(新加坡国立大学);ShanghaiTech University(上海科技大学);East China Normal University(华东师范大学);Nanjing University of Information Science & Technology(南京信息工程大学);Zhejiang University(浙江大学);Institute of Automation, Chinese Academy of Science(中国科学院自动化研究所);Shanghai AI Laboratory(上海人工智能实验室)
Visual Affordance Prediction: Survey and Reproducibility
Istituto Italiano di Tecnologia(意大利理工研究院);Centre for Intelligent Sensing(智能感知中心);Queen Mary University of London(伦敦女王学院);Idiap Research Institute(Idiap研究 institute);EPFL(瑞士联邦理工学院)
Learning Affordance Grounding from Exocentric Images
JD Explore Academy(京东探索研究院);University of Science and Technology of China(中国科学技术大学);Hefei Comprehensive National Science Center(合肥综合性国家科学中心人工智能研究院);The University of Sydney(悉尼大学)
Open-Vocabulary Affordance Detection in 3D Point Clouds
FPT Software AI Center(FPT软件人工智能中心);VNUHCM-University of Science(越南胡志明市国家大学-科学大学);Automation & Control Institute (ACIN), TU Wien(维也纳工业大学自动化与控制研究所);Center for Vision, Automation & Control, AIT Austrian Institute of Technology (GmbH)(奥地利技术研究院(AIT)视觉、自动化与控制中心);Ton Duc Thang University(孙德胜大学);University of Arkansas(阿肯色大学);University of Liverpool(利物浦大学)
University of Science and Technology of China(中国科学技术大学);The University of Sydney(悉尼大学);Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(合肥综合性国家科学中心人工智能研究院)