Zero-shot learning, the task of learning to recognize new classes not seen during training, has received considerable attention in the case of 2D image classification. However, despite the increasing ubiquity of 3D sensors, the corresponding 3D point cloud classification problem has not been meaningfully explored and introduces new challenges. In this paper, we identify some of the challenges and apply 2D Zero-Shot Learning (ZSL) methods in the 3D domain to analyze the performance of existing models. Then, we propose a novel approach to address the issues specific to 3D ZSL. We first present an inductive ZSL process and then extend it to the transductive ZSL and Generalized ZSL (GZSL) settings for 3D point cloud classification. To this end, a novel loss function is developed that simultaneously aligns seen semantics with point cloud features and takes advantage of unlabeled test data to address some known issues (e.g., the problems of domain adaptation, hubness, and data bias). While designed for the particularities of 3D point cloud classification, the method is shown to also be applicable to the more common use-case of 2D image classification. An extensive set of experiments is carried out, establishing state-of-the-art for ZSL and GZSL on synthetic (ModelNet40, ModelNet10, McGill) and real (ScanObjectNN) 3D point cloud datasets.
PointRFT: Explicit Reinforcement Fine-tuning for Point Cloud Few-shot Learning
School of Software Engineering, Xi’an Jiaotong University(西安交通大学软件工程学院);International School, Beijing University of Posts and Telecommunications(北京邮电大学国际学院)
Boosting Few-shot 3D Point Cloud Segmentation via Query-Guided Enhancement
Harbin Institute of Technology Shenzhen(哈尔滨工业大学深圳校区);SmartMore(思谋科技);Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies(广东省新型安全智能技术重点实验室)
ZeroReg: Zero-Shot Point Cloud Registration with Foundation Models
University of Trento(特伦托大学);Fondazione Bruno Kessler(布鲁诺·凯塞勒基金会);East China University of Science and Technology(华东理工大学);ETH Zurich(苏黎世联邦理工学院);Shanghai AI Laboratory(上海人工智能实验室)
Few-shot Class-incremental Learning for 3D Point Cloud Objects
North South University(北南大学);Australian National University(澳大利亚国立大学);Commonwealth Scientific and Industrial Research Organisation(澳大利亚联邦科学与工业研究组织);University of Adelaide(阿德莱德大学);The University of New South Wales(新南威尔士大学);University of Technology Sydney(悉尼科技大学)
Invariant Training 2D-3D Joint Hard Samples for Few-Shot Point Cloud Recognition
Nanyang Technological University(南洋理工大学);The University of Sydney(悉尼大学);Singapore Management University(新加坡管理大学);Terminus Group(特斯联科技集团);Institute for Infocomm Research(资讯通信研究院)
Zero-shot point cloud segmentation by transferring geometric primitives
The University of Hong Kong(香港大学);The Chinese University of Hong Kong(香港中文大学);Inceptio(嬴彻科技);ShanghaiTech University(上海科技大学);Texas A&M University(德州农工大学)
RARE: Refine Any Registration of Pairwise Point Clouds via Zero-Shot Learning
College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics(南京航空航天大学计算机科学与技术学院);Shenzhen Research Institute, Nanjing University of Aeronautics and Astronautics(南京航空航天大学深圳研究院);School of Data Science, Lingnan University(岭南大学数据科学学院)
PointDreamer: Zero-shot 3D Textured Mesh Reconstruction from Colored Point Cloud
Huazhong University of Science and Technology(华中科技大学);School of Computer Science and Technology(计算机科学与技术学院);Guangdong HUST Industrial Technology Research Institute(广东华中科技大学产业技术研究院);School of Computer Science and Engineering(计算机科学与工程学院);South China University of Technology(华南理工大学);Pazhou Laboratory(琶洲实验室)
PlantSegNeRF: A few-shot, cross-species method for plant 3D instance point cloud reconstruction via joint-channel NeRF with multi-view image instance matching
RW-Net: Enhancing Few-Shot Point Cloud Classification with a Wavelet Transform Projection-based Network
NYU Tandon School of Engineering, New York University(纽约大学坦登工程学院);NYUAD Center for Artificial Intelligence and Robotics, New York University Abu Dhabi(纽约大学阿布扎比分校人工智能与机器人中心)
CLIP2Point: Transfer CLIP to Point Cloud Classification with Image-Depth Pre-training
Harbin Institute of Technology(哈尔滨工业大学);City University of Hong Kong(香港城市大学);The University of Hong Kong(香港大学);Shanghai AI Laboratory(上海人工智能实验室);Peng Cheng Laboratory(鹏城实验室)
Foundation Model-Powered 3D Few-Shot Class Incremental Learning via Training-free Adaptor
University of Technology Sydney(悉尼科技大学);Data61, CSIRO(澳大利亚联邦科学与工业研究组织Data61);Australian National University(澳大利亚国立大学);North South University(南北大学);York University(约克大学)
本文提出一种基于3D基础模型且无需训练的适配器方法,利用双重缓存系统防止遗忘与过拟合,有效解决3D点云环境下的Few-Shot Class Incremental Learning问题,在多个数据集上表现优异。
Toward Zero-Shot Point Cloud Registration Across Diverse Scales, Scenes, and Sensor Setups
Laboratory for Information and Decision Systems (LIDS), Massachusetts Institute of Technology(信息与决策系统实验室,麻省理工学院);Department of Computer Science and Engineering, Seoul National University(计算机科学与工程系,首尔国立大学)