Identifying affordance regions on 3D objects from semantic cues is essential for robotics and human-machine interaction. However, existing 3D affordance learning methods struggle with generalization and robustness due to limited annotated data and a reliance on 3D backbones focused on geometric encoding, which often lack resilience to real-world noise and data corruption. We propose GEAL, a novel framework designed to enhance the generalization and robustness of 3D affordance learning by leveraging large-scale pre-trained 2D models. We employ a dual-branch architecture with Gaussian splatting to establish consistent mappings between 3D point clouds and 2D representations, enabling realistic 2D renderings from sparse point clouds. A granularity-adaptive fusion module and a 2D-3D consistency alignment module further strengthen cross-modal alignment and knowledge transfer, allowing the 3D branch to benefit from the rich semantics and generalization capacity of 2D models. To holistically assess the robustness, we introduce two new corruption-based benchmarks: PIAD-C and LASO-C. Extensive experiments on public datasets and our benchmarks show that GEAL consistently outperforms existing methods across seen and novel object categories, as well as corrupted data, demonstrating robust and adaptable affordance prediction under diverse conditions. Code and corruption datasets have been made publicly available.
Affordance-R1: Reinforcement Learning for Generalizable Affordance Reasoning in Multimodal Large Language Model
The Hong Kong University of Science and Technology (GZ)(香港科技大学(广州));National University of Singapore(新加坡国立大学);ShanghaiTech University(上海科技大学);East China Normal University(华东师范大学);Nanjing University of Information Science & Technology(南京信息工程大学);Zhejiang University(浙江大学);Institute of Automation, Chinese Academy of Science(中国科学院自动化研究所);Shanghai AI Laboratory(上海人工智能实验室)
Afford-X: Generalizable and Slim Affordance Reasoning for Task-oriented Manipulation
Institute for Artificial Intelligence, Peking University(北京大学人工智能研究院);Department of Computer Science and Engineering, Hong Kong University of Science and Technology(香港科技大学计算机科学与工程系);Shenyang Institute of Automation, Chinese Academy of Sciences(中国科学院沈阳自动化研究所);Institute for Al Industry Research, Tsinghua University(清华大学人工智能产业研究院);Department of Computer Science, Tsinghua University(清华大学计算机科学系)
FUNCanon: Learning Pose-Aware Action Primitives via Functional Object Canonicalization for Generalizable Robotic Manipulation
TAMS (Technical Aspects of Multimodal Systems), Department of Informatics, University of Hamburg(汉堡大学信息学院TAMS(多模态系统技术));Technical University of Munich(慕尼黑技术大学);Agile Robots SE(敏捷机器人有限公司)
MoMa-Kitchen: A 100K+ Benchmark for Affordance-Grounded Last-Mile Navigation in Mobile Manipulation
Fudan University(复旦大学);Shanghai AI Laboratory(上海人工智能实验室);University of Science and Technology of China(中国科学技术大学);Northwestern Polytechnical University(西北工业大学);TeleAI, China Telecom Corp Ltd(中国电信 TeleAI 公司)