We introduce a high-fidelity neural implicit dense visual Simultaneous Localization and Mapping (SLAM) system, termed DF-SLAM. In our work, we employ dictionary factors for scene representation, encoding the geometry and appearance information of the scene as a combination of basis and coefficient factors. Compared to neural implicit dense visual SLAM methods that directly encode scene information as features, our method exhibits superior scene detail reconstruction capabilities and more efficient memory usage, while our model size is insensitive to the size of the scene map, making our method more suitable for large-scale scenes. Additionally, we employ feature integration rendering to accelerate color rendering speed while ensuring color rendering quality, further enhancing the real-time performance of our neural SLAM method. Extensive experiments on synthetic and real-world datasets demonstrate that our method is competitive with existing state-of-the-art neural implicit SLAM methods in terms of real-time performance, localization accuracy, and scene reconstruction quality. Our source code is available at https://github.com/funcdecl/DF-SLAM.
Zhejiang University（浙江大学）；ETH Zurich（苏黎世联邦理工学院）；MPI for Intelligent Systems（马克斯·普朗克智能系统研究所）；Lund University（隆德大学）；University of Amsterdam（阿姆斯特丹大学）；Microsoft（微软）
Neural implicit representations have recently shown encouraging results in various domains, including promising progress in simultaneous localization and mapping (SLAM). Nevertheless, existing methods produce over-smoothed scene reconstructions and have difficulty scaling up to large scenes. These limitations are mainly due to their simple fully-connected network architecture that does not incorporate local information in the observations. In this paper, we present NICE-SLAM, a dense SLAM system that incorporates multi-level local information by introducing a hierarchical scene representation.…
ETH Zürich（苏黎世联邦理工学院）；MPI for Intelligent Systems（马克斯·普朗克智能系统研究所）；Lund University（隆德大学）；Zhejiang University（浙江大学）；University of Amsterdam（阿姆斯特丹大学）；University of Tübingen（图宾根大学）；Tübingen AI Center（图宾根人工智能中心）；Microsoft（微软）
Neural implicit representations have recently become popular in simultaneous localization and mapping (SLAM), especially in dense visual SLAM. However, previous works in this direction either rely on RGB-D sensors, or require a separate monocular SLAM approach for camera tracking and do not produce high-fidelity dense 3D scene reconstruction. In this paper, we present NICER-SLAM, a dense RGB SLAM system that simultaneously optimizes for camera poses and a hierarchical neural implicit map representation, which also allows for high-quality novel view synthesis. To facilitate the optimization pro…
DINO-SLAM: DINO-informed RGB-D SLAM for Neural Implicit and Explicit Representations
University of Bologna, Italy（博洛尼亚大学）；Faculty of Dentistry, The University of Hong Kong, China（香港大学牙科学院）；Rawmantic AI, China（Rawmantic AI）；Fudan University, Shanghai, China（复旦大学）
University of Bristol（布里斯托大学）；Zhejiang University（浙江大学）
We propose a novel end-to-end RGB-D SLAM, iDF-SLAM, which adopts a feature-based deep neural tracker as the front-end and a NeRF-style neural implicit mapper as the back-end. The neural implicit mapper is trained on-the-fly, while though the neural tracker is pretrained on the ScanNet dataset, it is also finetuned along with the training of the neural implicit mapper. Under such a design, our iDF-SLAM is capable of learning to use scene-specific features for camera tracking, thus enabling lifelong learning of the SLAM system. Both the training for the tracker and the mapper are self-supervised…
Fudan University（复旦大学）；Shanghai AI Laboratory（上海人工智能实验室）；Hong Kong University of Sciences and Technology（香港科技大学）；Shanghai Jiao Tong University（上海交通大学）；Northwestern Polytechnical University（西北工业大学）
DDN-SLAM: Real-time Dense Dynamic Neural Implicit SLAM
Dalian University of Technology（大连理工大学）；Saarland University of Applied Sciences（萨尔应用技术大学）；Shanghai Jiao Tong University（上海交通大学）；Dalian Maritime University（大连海事大学）
Alibaba Group（阿里巴巴集团）；Simon Fraser University（西蒙弗雷泽大学）；Hong Kong University of Science and Technology（香港科技大学）
There is an emerging trend of using neural implicit functions for map representation in Simultaneous Localization and Mapping (SLAM). Some pioneer works have achieved encouraging results on RGB-D SLAM. In this paper, we present a dense RGB SLAM method with neural implicit map representation. To reach this challenging goal without depth input, we introduce a hierarchical feature volume to facilitate the implicit map decoder. This design effectively fuses shape cues across different scales to facilitate map reconstruction. Our method simultaneously solves the camera motion and the neural implici…
We show for the first time that a multilayer perceptron (MLP) can serve as the only scene representation in a real-time SLAM system for a handheld RGB-D camera. Our network is trained in live operation without prior data, building a dense, scene-specific implicit 3D model of occupancy and colour which is also immediately used for tracking. Achieving real-time SLAM via continual training of a neural network against a live image stream requires significant innovation. Our iMAP algorithm uses a keyframe structure and multi-processing computation flow, with dynamic information-guided pixel samplin…
Representing visual signals by coordinate-based deep fully-connected networks has been shown advantageous in fitting complex details and solving inverse problems than discrete grid-based representation. However, acquiring such a continuous Implicit Neural Representation (INR) requires tedious per-scene training on tons of signal measurements, which limits its practicality. In this paper, we present a generic INR framework that achieves both data and training efficiency by learning a Neural Implicit Dictionary (NID) from a data collection and representing INR as a functional combination of basi…
Multi-Submap Implicit Neural SLAM with Local-to-Global Loop Closure for Large-Scale Scene Reconstruction
School of Automation and Intelligent Sensing, Shanghai Jiao Tong University（上海交通大学自动化与智能感知学院）；Carnegie Mellon University（卡内基梅隆大学）；School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电气与电子工程学院）
HI-SLAM: Monocular Real-time Dense Mapping with Hybrid Implicit Fields
University of Stuttgart（斯图加特大学）；Huawei Munich Research Center（华为慕尼黑研究中心）；Central Media Technology Institute, Huawei 2012 Laboratories（华为2012实验室中央媒体技术研究院）；Technical University of Munich（慕尼黑工业大学）
We present a novel explicit shape representation for instance segmentation. Based on how to model the object shape, current instance segmentation systems can be divided into two categories, implicit and explicit models. The implicit methods, which represent the object mask/contour by intractable network parameters, and produce it through pixel-wise classification, are predominant. However, the explicit methods, which parameterize the shape with simple and explainable models, are less explored. Since the operations to generate the final shape are light-weighted, the explicit methods have a clea…
ETH Zurich（苏黎世联邦理工学院）；École Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院）
Implicit neural representations (INRs) have recently emerged as a promising alternative to classical discretized representations of signals. Nevertheless, despite their practical success, we still do not understand how INRs represent signals. We propose a novel unified perspective to theoretically analyse INRs. Leveraging results from harmonic analysis and deep learning theory, we show that most INR families are analogous to structured signal dictionaries whose atoms are integer harmonics of the set of initial mapping frequencies. This structure allows INRs to express signals with an exponenti…
GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting
Shanghai AI Laboratory（上海人工智能实验室）；Hong Kong University of Science and Technology（香港科技大学）；Fudan University（复旦大学）；Northwestern Polytechnical University（西北工业大学）；TeleAI, China Telecom Corp Ltd（中国电信集团有限公司TeleAI）
In this paper, we consider the problems in the practical application of visual simultaneous localization and mapping (SLAM). With the popularization and application of the technology in wide scope, the practicability of SLAM system has become a new hot topic after the accuracy and robustness, e.g., how to keep the stability of the system and achieve accurate pose estimation in the low-texture and dynamic environment, and how to improve the universality and real-time performance of the system in the real scenes, etc. This paper proposes a real-time stereo indirect visual SLAM system, PLD-SLAM, …
National Taiwan University（国立台湾大学）；National Yang Ming Chiao Tung University（国立阳明交通大学）；Mobile Drive Technology（摩智科技）
A spatial AI that can perform complex tasks through visual signals and cooperate with humans is highly anticipated. To achieve this, we need a visual SLAM that easily adapts to new scenes without pre-training and generates dense maps for downstream tasks in real-time. None of the previous learning-based and non-learning-based visual SLAMs satisfy all needs due to the intrinsic limitations of their components. In this work, we develop a visual SLAM named Orbeez-SLAM, which successfully collaborates with implicit neural representation and visual odometry to achieve our goals. Moreover, Orbeez-SL…
LRSLAM: Low-rank Representation of Signed Distance Fields in Dense Visual SLAM System
Dept of Computer Science and Engineering, Korea University, Seoul, Korea（韩国大学计算机科学与工程系）；Dept of Electrical and Electronic Engineering, Yonsei University, Seoul, Korea（延世大学电气与电子工程系）；Meta Reality Labs, Pittsburgh, PA 15222, USA（Meta Reality Labs）
Dynamic Dictionary Learning for Remote Sensing Image Segmentation
School of Computer Science and Technology, Beijing Jiaotong University（北京交通大学计算机科学与技术学院）；Department of Computer Technology and Applications, Qinghai University（青海大学计算机技术与应用系）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）
OpenGS-SLAM: Open-Set Dense Semantic SLAM with 3D Gaussian Splatting for Object-Level Scene Understanding
School of Automation, Beijing Institute of Technology（北京理工大学自动化学院）；National Key Lab of Autonomous Intelligent Unmanned Systems, Beijing Institute of Technology（北京理工大学自主智能无人系统全国重点实验室）
Since the first success of Dong et al., the deep-learning-based approach has become dominant in the field of single-image super-resolution. This replaces all the handcrafted image processing steps of traditional sparse-coding-based methods with a deep neural network. In contrast to sparse-coding-based methods, which explicitly create high/low-resolution dictionaries, the dictionaries in deep-learning-based methods are implicitly acquired as a nonlinear combination of multiple convolutions. One disadvantage of deep-learning-based methods is that their performance is degraded for images created …
