SceneEncoder: Scene-Aware Semantic Segmentation of Point Clouds with A Learnable Scene Descriptor
Department of Computer Science and Engineering, Shanghai Jiao Tong University（上海交通大学计算机科学与工程系）；School of Computer Science and Software Engineering, East China Normal University（华东师范大学计算机科学与软件工程学院）
Jiachen Xu, Jingyu Gong, Jie Zhou, Xin Tan, Yuan Xie, Lizhuang Ma
Besides local features, global information plays an essential role in semantic segmentation, while recent works usually fail to explicitly extract the meaningful global information and make full use of it. In this paper, we propose a SceneEncoder module to impose a scene-aware guidance to enhance the effect of global information. The module predicts a scene descriptor, which learns to represent the categories of objects existing in the scene and directly guides the point-level semantic segmentation through filtering out categories not belonging to this scene. Additionally, to alleviate segmentation noise in local region, we design a region similarity loss to propagate distinguishing features to their own neighboring points with the same label, leading to the enhancement of the distinguishing ability of point-wise features. We integrate our methods into several prevailing networks and conduct extensive experiments on benchmark datasets ScanNet and ShapeNet. Results show that our methods greatly improve the performance of baselines and achieve state-of-the-art performance.
We introduce modifications to state-of-the-art approaches to aggregating local video descriptors by using attention mechanisms and function approximations. Rather than using ensembles of existing architectures, we provide an insight on creating new architectures. We demonstrate our solutions in the "The 2nd YouTube-8M Video Understanding Challenge", by using frame-level video and audio descriptors. We obtain testing accuracy similar to the state of the art, while meeting budget constraints, and touch upon strategies to improve the state of the art. Model implementations are available in https:…
Learnable keypoint detectors and descriptors are beginning to outperform classical hand-crafted feature extraction methods. Recent studies on self-supervised learning of visual representations have driven the increasing performance of learnable models based on deep networks. By leveraging traditional data augmentations and homography transformations, these networks learn to detect corners under adverse conditions such as extreme illumination changes. However, their generalization capabilities are limited to corner-like features detected a priori by classical methods or synthetically generated …
MSSPlace: Multi-Sensor Place Recognition with Visual and Text Semantics
Intelligent Transport Laboratory, Moscow Institute of Physics and Technology（智能交通实验室，莫斯科物理技术学院）；Artificial Intelligence Research Institute (AIRI)（人工智能研究机构（AIRI））
Samsung AI Center, Moscow, Russia（三星人工智能中心（莫斯科））；Skolkovo Institute of Science and Technology (Skoltech), Moscow, Russia（斯科尔科沃科学技术研究院（莫斯科））
We propose and evaluate a neural point-based graphics method that can model semi-transparent scene parts. Similarly to its predecessor pipeline, ours uses point clouds to model proxy geometry, and augments each point with a neural descriptor. Additionally, a learnable transparency value is introduced in our approach for each point. Our neural rendering procedure consists of two steps. Firstly, the point cloud is rasterized using ray grouping into a multi-channel image. This is followed by the neural rendering step that "translates" the rasterized image into an RGB output using a learnable conv…
Samsung AI Center（三星人工智能中心）；Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究所）；In3D.io
We present a new point-based approach for modeling the appearance of real scenes. The approach uses a raw point cloud as the geometric representation of a scene, and augments each point with a learnable neural descriptor that encodes local geometry and appearance. A deep rendering network is learned in parallel with the descriptors, so that new views of the scene can be obtained by passing the rasterizations of a point cloud from new viewpoints through this network. The input rasterizations use the learned descriptors as point pseudo-colors. We show that the proposed approach can be used for m…
FBK（布鲁诺·凯斯勒基金会）；University of Trento（特伦托大学）；Universitat de Barcelona（巴塞罗那大学）；Computer Vision Center（计算机视觉中心）
We present EgoACO, a deep neural architecture for video action recognition that learns to pool action-context-object descriptors from frame level features by leveraging the verb-noun structure of action labels in egocentric video datasets. The core component of EgoACO is class activation pooling (CAP), a differentiable pooling operation that combines ideas from bilinear pooling for fine-grained recognition and from feature learning for discriminative localization. CAP uses self-attention with a dictionary of learnable weights to pool from the most relevant feature regions. Through CAP, EgoACO …
New York University Abu Dhabi（纽约大学阿布扎比分校）；NYU Tandon School of Engineering（纽约大学坦登工程学院）
In this paper, we propose a novel method named GP-Aligner to deal with the problem of non-rigid groupwise point set registration. Compared to previous non-learning approaches, our proposed method gains competitive advantages by leveraging the power of deep neural networks to effectively and efficiently learn to align a large number of highly deformed 3D shapes with superior performance. Unlike most learning-based methods that use an explicit feature encoding network to extract the per-shape features and their correlations, our model leverages a model-free learnable latent descriptor to charact…
Towards Implicit Aggregation: Robust Image Representation for Place Recognition in the Transformer Era
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院，清华大学）；Pengcheng Laboratory（鹏城实验室）；Shenyang Institute of Automation, Chinese Academy of Sciences（沈阳自动化研究所，中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）；Pazhou Laboratory (Huangpu)（琶洲实验室（黄埔））
Unlike standard object classification, where the image to be classified contains one or multiple instances of the same object, indoor scene classification is quite different since the image consists of multiple distinct objects. Further, these objects can be of varying sizes and are present across numerous spatial locations in different layouts. For automatic indoor scene categorization, large scale spatial layout deformations and scale variations are therefore two major challenges and the design of rich feature descriptors which are robust to these challenges is still an open problem. This pa…
Few-shot Shape Recognition by Learning Deep Shape-aware Features
Southeast University（东南大学）；The Australian National University（澳大利亚国立大学）；University of Massachusetts, Dartmouth（马萨诸塞大学达特茅斯分校）；Data61/CSIRO（Data61/联邦科学与工业研究组织）
CLIDD: Cross-Layer Independent Deformable Description for Efficient and Discriminative Local Feature Representation
School of Astronautics, Harbin Institute of Technology, China（航天学院，哈尔滨工业大学，中国）；State Key Laboratory of General Artificial Intelligence, Beijing Institute for General Artificial Intelligence (BIGAI), China（通用人工智能国家重点实验室，北京通用人工智能研究院（BIGAI），中国）
We address a core problem of computer vision: Detection and description of 2D feature points for image matching. For a long time, hand-crafted designs, like the seminal SIFT algorithm, were unsurpassed in accuracy and efficiency. Recently, learned feature detectors emerged that implement detection and description using neural networks. Training these networks usually resorts to optimizing low-level matching scores, often pre-defining sets of image patches which should or should not match, or which should or should not contain key points. Unfortunately, increased accuracy for these low-level ma…
Memory-Guided Point Cloud Completion for Dental Reconstruction
College of Information and Intelligent Science, Donghua University, Shanghai, China（信息与智能科学学院，东华大学，上海，中国）；Institute of Artificial Intelligence, Shanghai University, Shanghai, China（人工智能研究院，上海大学，上海，中国）
NYU Multimedia and Visual Computing Lab（纽约大学多媒体与视觉计算实验室）；New York University Abu Dhabi（纽约大学阿布扎比分校）；New York University（纽约大学）
In this paper, we deal with the problem to predict the future 3D motions of 3D object scans from previous two consecutive frames. Previous methods mostly focus on sparse motion prediction in the form of skeletons. While in this paper we focus on predicting dense 3D motions in the from of 3D point clouds. To approach this problem, we propose a self-supervised approach that leverages the power of the deep neural network to learn a continuous flow function of 3D point clouds that can predict temporally consistent future motions and naturally bring out the correspondences among consecutive point c…
SVIP: Semantically Contextualized Visual Patches for Zero-Shot Learning
University of Southern Queensland（南昆士兰大学）；University of Queensland（昆士兰大学）；The Hong Kong Polytechnic University（香港理工大学）；University of Electronic Science and Technology of China（中国电子科学技术研究中心）
FedHPro: Federated Hyper-Prototype Learning via Gradient Matching
School of Computing and Information Technology, University of Wollongong, Wollongong, Australia（计算机与信息科技学院，沃尔灵戈大学，澳大利亚）；School of Computing and Information Systems, Singapore Management University, Singapore, Singapore（计算机与信息系统学院，新加坡管理大学，新加坡）；Monash University, Australia（莫纳什大学，澳大利亚）；Macquarie University, Australia（麦考瑞大学，澳大利亚）；Beijing Institute of Technology, China（北京理工大学，中国）；La Trobe University, Australia（拉筹伯大学，澳大利亚）
GLIDE-Reg: Global-to-Local Deformable Registration Using Co-Optimized Foundation and Handcrafted Features
1 Medical \& Imaging Informatics, Department of Radiological Sciences, David Geffen School of Medicine at UCLA, USA 2 Magnetic Resonance Research Labs, Department of Radiological Sciences, David Geffen School of Medicine at UCLA, USA 3 Department of Radiation Oncology, David Geffen School of Medicine at UCLA, Los Angeles, USA
