Automated Parking is becoming a standard feature in modern vehicles. Existing parking systems build a local map to be able to plan for maneuvering towards a detected slot. Next generation parking systems have an use case where they build a persistent map of the environment where the car is frequently parked, say for example, home parking or office parking. The pre-built map helps in re-localizing the vehicle better when its trying to park the next time. This is achieved by augmenting the parking system with a Visual SLAM pipeline and the feature is called trained trajectory parking in the automotive industry. In this paper, we discuss the use cases, design and implementation of a trained trajectory automated parking system. The proposed system is deployed on commercial vehicles and the consumer application is illustrated in \url{https://youtu.be/nRWF5KhyJZU}. The focus of this paper is on the application and the details of vision algorithms are kept at high level.
Fudan University（复旦大学）；FAW (Nanjing) Technology Development Co.,Ltd（一汽（南京）科技开发有限公司）
Automatic underground parking has attracted considerable attention as the scope of autonomous driving expands. The auto-vehicle is supposed to obtain the environmental information, track its location, and build a reliable map of the scenario. Mainstream solutions consist of well-trained neural networks and simultaneous localization and mapping (SLAM) methods, which need numerous carefully labeled images and multiple sensor estimations. However, there is a lack of underground parking scenario datasets with multiple sensors and well-labeled images that support both SLAM tasks and perception task…
Towards Autonomous Indoor Parking: A Globally Consistent Semantic SLAM System and A Semantic Localization Subsystem
School of Automation and Intelligent Sensing, Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai Jiao Tong University, Shanghai（自动化与智能感知学院、系统控制与信息处理重点实验室、中华人民共和国教育部、上海交通大学、上海）
Hilti AG（喜利得公司）；University of Zurich（苏黎世大学）；ETH Zurich（苏黎世联邦理工学院）
Research in Simultaneous Localization and Mapping (SLAM) has made outstanding progress over the past years. SLAM systems are nowadays transitioning from academic to real world applications. However, this transition has posed new demanding challenges in terms of accuracy and robustness. To develop new SLAM systems that can address these challenges, new datasets containing cutting-edge hardware and realistic scenarios are required. We propose the Hilti SLAM Challenge Dataset. Our dataset contains indoor sequences of offices, labs, and construction environments and outdoor sequences of constructi…
NDT-Map-Code: A 3D global descriptor for real-time loop closure detection in lidar SLAM
The College of Mechanical and Vehicle Engineering, Chongqing University（重庆大学机械与车辆工程学院）；NIO（蔚来）；The State Key Lab of Mechanical Transmissions, Chongqing University（重庆大学机械传动国家重点实验室）；Department of Computer Science, The University of Sheffield（谢菲尔德大学计算机科学系）
Autonomous valet parking is a specific application for autonomous vehicles. In this task, vehicles need to navigate in narrow, crowded and GPS-denied parking lots. Accurate localization ability is of great importance. Traditional visual-based methods suffer from tracking lost due to texture-less regions, repeated structures, and appearance changes. In this paper, we exploit robust semantic features to build the map and localize vehicles in parking lots. Semantic features contain guide signs, parking lines, speed bumps, etc, which typically appear in parking lots. Compared with traditional feat…
Simultaneous mapping and localization (SLAM) in an real indoor environment is still a challenging task. Traditional SLAM approaches rely heavily on low-level geometric constraints like corners or lines, which may lead to tracking failure in textureless surroundings or cluttered world with dynamic objects. In this paper, a compact semantic SLAM framework is proposed, with utilization of both geometric and object-level semantic constraints jointly, a more consistent mapping result, and more accurate pose estimation can be obtained. Two main contributions are presented int the paper, a) a robust …
Simultaneous Localization & Mapping (SLAM) is the process of building a mutual relationship between localization and mapping of the subject in its surrounding environment. With the help of different sensors, various types of SLAM systems have developed to deal with the problem of building the relationship between localization and mapping. A limitation in the SLAM process is the lack of consideration of dynamic objects in the mapping of the environment. We propose the Dynamic Object Tracking SLAM (DyOb-SLAM), which is a Visual SLAM system that can localize and map the surrounding dynamic object…
Instituto de Investigación en Ingeniería de Aragón (I3A)（阿拉贡工程研究所（I3A））；Universidad de Zaragoza（萨拉戈萨大学）
The assumption of scene rigidity is common in visual SLAM algorithms. However, it limits their applicability in populated real-world environments. Furthermore, most scenarios including autonomous driving, multi-robot collaboration and augmented/virtual reality, require explicit motion information of the surroundings to help with decision making and scene understanding. We present in this paper DynaSLAM II, a visual SLAM system for stereo and RGB-D configurations that tightly integrates the multi-object tracking capability. DynaSLAM II makes use of instance semantic segmentation and of ORB feat…
CCDC Army Research Laboratory（美国陆军作战能力发展司令部陆军研究实验室）；University of Oxford（牛津大学）；Monash University（莫纳什大学）；University of Edinburgh（爱丁堡大学）；University of Cambridge（剑桥大学）
Simultaneous Localization and Mapping (SLAM) system typically employ vision-based sensors to observe the surrounding environment. However, the performance of such systems highly depends on the ambient illumination conditions. In scenarios with adverse visibility or in the presence of airborne particulates (e.g. smoke, dust, etc.), alternative modalities such as those based on thermal imaging and inertial sensors are more promising. In this paper, we propose the first complete thermal-inertial SLAM system which combines neural abstraction in the SLAM front end with robust pose graph optimizatio…
RootSLAM: Square Root Bundle Adjustment for Visual-Inertial SLAM
Technical University of Munich（慕尼黑工业大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）；National University of Rosario（罗萨里奥国立大学）；CIFASIS (CONICET-UNR)（CIFASIS（CONICET-UNR））
School of Integrated Circuits, Tsinghua University（清华大学集成电路学院）；Tsinghua University（清华大学）
Simultaneous localization and mapping (SLAM) is a critical technology that enables autonomous robots to be aware of their surrounding environment. With the development of deep learning, SLAM systems can achieve a higher level of perception of the environment, including the semantic and text levels. However, current works are limited in their ability to achieve a natural-language level of perception of the world. To address this limitation, we propose LP-SLAM, the first language-perceptive SLAM system that leverages large language models (LLMs). LP-SLAM has two major features: (a) it can detect…
Delivery robots aim to achieve high precision to facilitate complete autonomy. A precise three-dimensional point cloud map of sidewalk surroundings is required to estimate self-location. With or without the loop closing method, the cumulative error increases gradually after mapping for larger urban or city maps due to sensor drift. Therefore, there is a high risk of using the drifted or misaligned map. This article presented a technique for fusing GPS to update the 3D point cloud and eliminate cumulative error. The proposed method shows outstanding results in quantitative comparison and qualit…
Simultaneous Localization and Mapping (SLAM) is considered to be a fundamental capability for intelligent mobile robots. Over the past decades, many impressed SLAM systems have been developed and achieved good performance under certain circumstances. However, some problems are still not well solved, for example, how to tackle the moving objects in the dynamic environments, how to make the robots truly understand the surroundings and accomplish advanced tasks. In this paper, a robust semantic visual SLAM towards dynamic environments named DS-SLAM is proposed. Five threads run in parallel in DS-…
The automatic parking is being massively developed by car manufacturers and providers. Until now, there are two problems with the automatic parking. First, there is no openly-available segmentation labels of parking slot on panoramic surround view (PSV) dataset. Second, how to detect parking slot and road structure robustly. Therefore, in this paper, we build up a public PSV dataset. At the same time, we proposed a highly fused convolutional network (HFCN) based segmentation method for parking slot and lane markings based on the PSV dataset. A surround-view image is made of four calibrated ima…
A Simultaneous Localization and Mapping (SLAM) system must be robust to support long-term mobile vehicle and robot applications. However, camera and LiDAR based SLAM systems can be fragile when facing challenging illumination or weather conditions which degrade their imagery and point cloud data. Radar, whose operating electromagnetic spectrum is less affected by environmental changes, is promising although its distinct sensing geometry and noise characteristics bring open challenges when being exploited for SLAM. % However, there are still open challenges since most existing visual and LiDAR …
We equip an ultra-wideband (UWB) node and a 2D LiDAR sensor a.k.a. 2D laser rangefinder on a mobile robot, and place UWB beacon nodes at unknown locations in an unknown environment. All UWB nodes can do ranging with each other thus forming a cooperative sensor network. We propose to fuse the peer-to-peer ranges measured between UWB nodes and laser scanning information, i.e. range measured between robot and nearby objects/obstacles, for simultaneous localization of the robot, all UWB beacons, and LiDAR mapping. The fusion is inspired by two facts: 1) LiDAR may improve UWB-only localization accu…
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）
In feature-based simultaneous localization and mapping (SLAM), line features complement the sparsity of point features, making it possible to map the surrounding environment structure. Existing approaches utilizing line features have primarily employed a measurement model that uses line re-projection. However, the direction vectors used in the 3D line mapping process cannot be corrected because the line measurement model employs only the lines' normal vectors in the Plücker coordinate. As a result, problems like degeneracy that occur during the 3D line mapping process cannot be solved. To tack…
University at Buffalo（布法罗大学）；University of Southern California（南加利福尼亚大学）
Recent trends have accelerated the development of spatial applications on mobile devices and robots. These include navigation, augmented reality, human-robot interaction, and others. A key enabling technology for such applications is the understanding of the device's location and the map of the surrounding environment. This generic problem, referred to as Simultaneous Localization and Mapping (SLAM), is an extensively researched topic in robotics. However, visual SLAM algorithms face several challenges including perceptual aliasing and high computational cost. These challenges affect the accur…
DYNEMO-SLAM: Dynamic Entity and Motion-Aware 3D Scene Graph SLAM
Automation and Robotics Research Group, Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg（卢森堡大学自动化与机器人研究组，安全、可靠与信任跨学科研究中心（SnT））
Technical University of Munich（慕尼黑工业大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；Google Inc.（谷歌公司）
This work proposes a RGB-D SLAM system specifically designed for structured environments and aimed at improved tracking and mapping accuracy by relying on geometric features that are extracted from the surrounding. Structured environments offer, in addition to points, also an abundance of geometrical features such as lines and planes, which we exploit to design both the tracking and mapping components of our SLAM system. For the tracking part, we explore geometric relationships between these features based on the assumption of a Manhattan World (MW). We propose a decoupling-refinement method b…
Autonomous navigation requires an accurate model or map of the environment. While dramatic progress in the prior two decades has enabled large-scale SLAM, the majority of existing methods rely on non-linear optimization techniques to find the MLE of the robot trajectory and surrounding environment. These methods are prone to local minima and are thus sensitive to initialization. Several recent papers have developed optimization algorithms for the Pose-Graph SLAM problem that can certify the optimality of a computed solution. Though this does not guarantee a priori that this approach generates …
NANO-SLAM : Natural Gradient Gaussian Approximation for Vehicle SLAM
School of Vehicle and Mobility, Tsinghua University（车辆与移动性学院，清华大学）；Department of Advanced Manufacturing and Robotics, College of Engineering, Peking University（先进制造与机器人系，工程学院，北京大学）；School of Vehicle and Mobility and College of Artificial Intelligence, Tsinghua University（车辆与移动性学院和人工智能学院，清华大学）
针对车辆SLAM中现有RBPF方法因线性化引入近似误差的问题，提出NANO-SLAM，通过自然梯度下降求解高斯参数优化问题来优化采样分布，在Sydney Victoria Park数据集上定位精度提升超50%且计算成本极低。
NanoSLAM: Enabling Fully Onboard SLAM for Tiny Robots
Integrated Systems Laboratory of ETH Zürich（苏黎世联邦理工学院集成系统实验室）；Center for Project-Based Learning of ETH Zürich（苏黎世联邦理工学院项目导向学习中心）；Department of Electrical, Electronic and Information Engineering of University of Bologna（博洛尼亚大学电子、电子与信息工程系）
MA-SLAM: Active SLAM in Large-Scale Unknown Environment using Map Aware Deep Reinforcement Learning
Sun Yat-sen University（中山大学）；South China University of Technology（华南理工大学）；Hong Kong University of Science and Technology（香港科学与技术大学）；City University of Hong Kong（香港城市大学）
Automated Parking is a low speed manoeuvring scenario which is quite unstructured and complex, requiring full 360° near-field sensing around the vehicle. In this paper, we discuss the design and implementation of an automated parking system from the perspective of camera based deep learning algorithms. We provide a holistic overview of an industrial system covering the embedded system, use cases and the deep learning architecture. We demonstrate a real-time multi-task deep learning network called FisheyeMultiNet, which detects all the necessary objects for parking on a low-power embedded syste…
Hong Kong Polytechnic University（香港理工大学）；Nanjing University of Science and Technology（南京理工大学）
LiDAR-based SLAM algorithms are extensively studied to providing robust and accurate positioning for autonomous driving vehicles (ADV) in the past decades. Satisfactory performance can be obtained using high-grade 3D LiDAR with 64 channels, which can provide dense point clouds. Unfortunately, the high price significantly prevents its extensive commercialization in ADV. The cost-effective 3D LiDAR with 16 channels is a promising replacement. However, only limited and sparse point clouds can be provided by the 16 channels LiDAR, which cannot guarantee sufficient positioning accuracy for ADV in c…
Zongmu Technology（纵目科技）；Huazhong University of Science and Technology（华中科技大学）；Technical University of Braunschweig（布伦瑞克工业大学）
Surround-view fisheye perception under valet parking scenes is fundamental and crucial in autonomous driving. Environmental conditions in parking lots perform differently from the common public datasets, such as imperfect light and opacity, which substantially impacts on perception performance. Most existing networks based on public datasets may generalize suboptimal results on these valet parking scenes, also affected by the fisheye distortion. In this article, we introduce a new large-scale fisheye dataset called Fisheye Parking Dataset(FPD) to promote the research in dealing with diverse re…
NightOwl is a robotic platform designed exclusively for a wheeled service robot. The robot navigates autonomously in omnidirectional fashion movement and equipped with LIDAR to sense the surrounding area. The platform itself was built using the Robot Operating System (ROS) and written in two different programming languages (C++ and Python). NightOwl is composed of several modular programs, namely hardware controller, light detection and ranging (LIDAR), simultaneous localization and mapping (SLAM), world model, path planning, robot control, communication, and behaviour. The programs run in par…
Singapore University of Technology and Design（新加坡科技设计大学）；Nanyang Technological University（南洋理工大学）；University of Moratuwa（莫拉图瓦大学）
Simultaneous localization and mapping (SLAM) has been richly researched in past years particularly with regard to range-based or visual-based sensors. Instead of deploying dedicated devices that use visual features, it is more pragmatic to exploit the radio features to achieve this task, due to their ubiquitous nature and the wide deployment of Wifi wireless network. In this paper, we present a novel approach for crowd-sensing simultaneous localization and radio fingerprint mapping (C-SLAM-RF) in large unknown indoor environments. The proposed system makes use of the received signal strength (…
University of Applied Science, Gelsenkirchen（盖尔森基兴应用科学大学）；German Rescue Robotic Centre, Dortmund（多特蒙德德国救援机器人中心）；Fire Department of Dortmund, Institute of Fire Service and Rescue Technology（多特蒙德消防局消防与救援技术研究所）
This technical report is about the mission and the experience gained during the reconnaissance of an industrial hall with hazardous substances after a major fire in Berlin. During this operation, only UAVs and cameras were used to obtain information about the site and the building. First, a geo-referenced 3D model of the building was created in order to plan the entry into the hall. Subsequently, the UAVs were used to fly in the heavily damaged interior and take pictures from inside of the hall. A 360° camera mounted under the UAV was used to collect images of the surrounding area especially f…
Efficient Submap-based Autonomous MAV Exploration using Visual-Inertial SLAM Configurable for LiDARs or Depth Cameras
Technical University of Munich（慕尼黑技术大学）；School of Computation, Information and Technology（计算、信息与技术学院）；Imperial College London（伦敦帝国学院）；Munich Institute of Robotics and Machine Intelligence（慕尼黑机器人与机器智能研究所）；Munich Center for Machine Learning（慕尼黑机器学习中心）
In this paper, we present a multi-camera visual odometry (VO) system for an autonomous vehicle. Our system mainly consists of a virtual LiDAR and a pose tracker. We use a perspective transformation method to synthesize a surround-view image from undistorted fisheye camera images. With a semantic segmentation model, the free space can be extracted. The scans of the virtual LiDAR are generated by discretizing the contours of the free space. As for the pose tracker, we propose a visual odometry system fusing both the feature matching and the virtual LiDAR scan matching results. Only those feature…
Instituto de Investigación en Ingeniería de Aragón (I3A)（阿拉贡工程研究院（I3A））；Universidad de Zaragoza（萨拉戈萨大学）；Massachusetts Institute of Technology（麻省理工学院）；Genius Sports（Genius体育）；UC San Diego（加州大学圣地亚哥分校）；Technion – Israel Institute of Technology（以色列理工学院）
Active Simultaneous Localization and Mapping (SLAM) is the problem of planning and controlling the motion of a robot to build the most accurate and complete model of the surrounding environment. Since the first foundational work in active perception appeared, more than three decades ago, this field has received increasing attention across different scientific communities. This has brought about many different approaches and formulations, and makes a review of the current trends necessary and extremely valuable for both new and experienced researchers. In this work, we survey the state-of-the-a…
LIMOT: A Tightly-Coupled System for LiDAR-Inertial Odometry and Multi-Object Tracking
Tongji University（同济大学）；Institute of Intelligent Vehicles, Tongji University（同济大学智能车辆研究所）；School of Electronics and Information Engineering, Tongji University（同济大学电子与信息工程学院）；School of Surveying and Geo-Informatics, Tongji University（同济大学测绘与地理信息学院）
Determining the position and orientation of a sensor vis-a-vis its surrounding, while simultaneously mapping the environment around that sensor or simultaneous localization and mapping is quickly becoming an important advancement in embedded vision with a large number of different possible applications. This paper presents a comprehensive comparison of the latest open-source SLAM algorithms with the main focus being their performance in different environmental surroundings. The chosen algorithms are evaluated on common publicly available datasets and the results reasoned with respect to the da…
Handong Global University（韩东国际大学）；VADAS Co., Ltd.（VADAS有限公司）
This paper proposes an efficient autonomous valet parking system utilizing only cameras which are the most widely used sensor. To capture more information instantaneously and respond rapidly to changes in the surrounding environment, fisheye cameras which have a wider angle of view compared to pinhole cameras are used. Accordingly, visual simultaneous localization and mapping is used to identify the layout of the parking lot and track the location of the vehicle. In addition, the input image frames are converted into around view monitor images to resolve the distortion of fisheye lens because …
We propose a Visual Teach and Repeat (VTR) algorithm using semantic landmarks extracted from environmental objects for ground robots with fixed mount monocular cameras. The proposed algorithm is robust to changes in the starting pose of the camera/robot, where a pose is defined as the planar position plus the orientation around the vertical axis. VTR consists of a teach phase in which a robot moves in a prescribed path, and a repeat phase in which the robot tries to repeat the same path starting from the same or a different pose. Most available VTR algorithms are pose dependent and cannot perf…
COMPASS: COmpact Multi-channel Prior-map And Scene Signature for Floor-Plan-Based Visual Localization
Automation and Robotics Research Group, Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg (UL)（自动化与机器人研究组，安全、可靠性与信任跨学科中心（SnT），卢森堡大学（UL））
