It has witnessed a growing demand for efficient representation learning on point clouds in many 3D computer vision applications. Behind the success story of convolutional neural networks (CNNs) is that the data (e.g., images) are Euclidean structured. However, point clouds are irregular and unordered. Various point neural networks have been developed with isotropic filters or using weighting matrices to overcome the structure inconsistency on point clouds. However, isotropic filters or weighting matrices limit the representation power. In this paper, we propose a permutable anisotropic convolutional operation (PAI-Conv) that calculates soft-permutation matrices for each point using dot-product attention according to a set of evenly distributed kernel points on a sphere's surface and performs shared anisotropic filters. In fact, dot product with kernel points is by analogy with the dot-product with keys in Transformer as widely used in natural language processing (NLP). From this perspective, PAI-Conv can be regarded as the transformer for point clouds, which is physically meaningful and is robust to cooperate with the efficient random point sampling method. Comprehensive experiments on point clouds demonstrate that PAI-Conv produces competitive results in classification and semantic segmentation tasks compared to state-of-the-art methods.
3D Anisotropic Hybrid Network: Transferring Convolutional Features from 2D Images to 3D Anisotropic Volumes
Siemens Healthineers, X-ray products(西门子医疗 X 射线产品部);Siemens Healthineers, Medical Imaging Technologies(西门子医疗医学影像技术部);School of Information Technologies, University of Sydney(悉尼大学信息技术学院)
Istituto Nazionale di Ottica del Consiglio Nazionale delle Ricerche (CNR-INO)(意大利国家光学研究所(国家研究委员会光学研究所));University of Florence(佛罗伦萨大学);European Laboratory for Non-Linear Spectroscopy (LENS)(欧洲非线性光谱实验室)
Learning Local Neighboring Structure for Robust 3D Shape Representation
MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University(上海交通大学人工智能研究院教育部人工智能重点实验室);Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University(上海交通大学图像通信与网络工程研究所);School of Mathematical Sciences, University of Science and Technology of China(中国科学技术大学数学科学学院)