THUD++: Large-Scale Dynamic Indoor Scene Dataset and Benchmark for Mobile Robots
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）
Most existing mobile robotic datasets primarily capture static scenes, limiting their utility for evaluating robotic performance in dynamic environments. To address this, we present a mobile robot oriented large-scale indoor dataset, denoted as THUD++ (TsingHua University Dynamic) robotic dataset, for dynamic scene understanding. Our current dataset includes 13 large-scale dynamic scenarios, combining both real-world and synthetic data collected with a real robot platform and a physical simulation platform, respectively. The RGB-D dataset comprises over 90K image frames, 20M 2D/3D bounding boxes of static and dynamic objects, camera poses, and IMU. The trajectory dataset covers over 6,000 pedestrian trajectories in indoor scenes. Additionally, the dataset is augmented with a Unity3D-based simulation platform, allowing researchers to create custom scenes and test algorithms in a controlled environment. We evaluate state-of-the-art methods on THUD++ across mainstream indoor scene understanding tasks, e.g., 3D object detection, semantic segmentation, relocalization, pedestrian trajectory prediction, and navigation. Our experiments highlight the challenges mobile robots encounter in indoor environments, especially when navigating in complex, crowded, and dynamic scenes. By sharing this dataset, we aim to accelerate the development and testing of mobile robot algorithms, contributing to real-world robotic applications.
HY Technic Ltd. Sti.（HY技术有限公司）；Eskisehir Osmangazi University（埃斯基谢希尔奥斯曼加齐大学）；Inovasyon Muhendislik Ltd. Sti.（创新工程有限公司）
Mobile robots have the capability to work in real-time autonomously. Autonomous behavior is strictly dependent on knowing the position of the mobile robot. The positioning of a mobile robot in an indoor area is a difficult task for only one sensor information is used. We proposed a system and method to locate the mobile robot via fusing signals from WIFI and odometer data via particle filter. In this study, the Particle filter is a well-known filter that is used for indoor positioning of mobile robots. The proposed system includes two parts that are RFKON system and evarobot for data collectio…
Zen Aerologiks；BMS College of Engineering（BMS工程学院）
The navigation system is at the heart of any mobile robot it comprises of SLAM and path planning units, which is utilized by the robot to generate a map of the environment, localize itself within it and determine an optimal a path to the destination. This paper describes the conceptualization, development, simulation and hardware implementation of GenNav a generic indoor navigation system for any mobile aerial or ground robot. The generalization is brought about by modularizing and creating independence between the software computation and hardware actuation units by providing an alternate sou…
Although indoor localization has been a wide researched topic, obtained results may not fit the requirements that some domains need. Most approaches are not able to precisely localize a fast moving object even with a complex installation, which makes their implementation in the automated driving domain complicated. In this publication, common technologies were analyzed and a commercial product, called Marvelmind Indoor GPS, was chosen for our use case in which both ultrasound and radio frequency communications are used. The evaluation is given in a first moment on small indoor scenarios with s…
The CREATE database is composed of 14 hours of multimodal recordings from a mobile robotic platform based on the iRobot Create. The various sensors cover vision, audition, motors and proprioception. The dataset has been designed in the context of a mobile robot that can learn multimodal representations of its environment, thanks to its ability to navigate the environment. This ability can also be used to learn the dependencies and relationships between the different modalities of the robot (e.g. vision, audition), as they reflect both the external environment and the internal state of the robo…
The Chinese University of Hong Kong（香港中文大学）；Southern University of Science and Technology（南方科技大学）；Shenzhen Research Institute of the Chinese University of Hong Kong（香港中文大学深圳研究院）
As one of the most promising areas, mobile robots draw much attention these years. Current work in this field is often evaluated in a few manually designed scenarios, due to the lack of a common experimental platform. Meanwhile, with the recent development of deep learning techniques, some researchers attempt to apply learning-based methods to mobile robot tasks, which requires a substantial amount of data. To satisfy the underlying demand, in this paper we build HouseExpo, a large-scale indoor layout dataset containing 35,126 2D floor plans including 252,550 rooms in total. Together we develo…
Localization of a wireless mobile device or a robot in indoor and GPS-denied environments is a difficult problem, particularly in dynamic scenarios where traditional cameras and LIDAR-based alternative sensing and localization modalities may fail. We propose a method for estimating the location of a mobile robot in relation to static wireless sensor nodes (WSN) deployed in the environment. The method employs a novel particle filter that updates its weights using a Gauss probability over Direction of Arrival (DOA) estimate in conjunction with the mobile robot's mobility model. We evaluate and v…
This paper proposes a framework to measure the important metrics (throughput, delay, packet retransmits, signal strength, etc.) to determine Wi-Fi network performance of mobile robots supported by the Robot Operating Systems (ROS) middleware. We analyze the bidirectional network performance of mobile robots through an experimental setup in an indoor environment, where a mobile robot is communicating vital sensor data such as video streaming from the camera(s) and LiDAR scan values to a command station while it navigates an indoor environment through teleoperated velocity commands received from…
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）
The paper focuses on the development of the autonomous robot UltraBot to reduce COVID-19 transmission and other harmful bacteria and viruses. The motivation behind the research is to develop such a robot that is capable of performing disinfection tasks without the use of harmful sprays and chemicals that can leave residues, require airing the room afterward for a long time, and can cause the corrosion of the metal structures. UltraBot technology has the potential to offer the most optimal autonomous disinfection performance along with taking care of people, keeping them from getting under UV-C…
University Libre du Bruxelles（布鲁塞尔自由大学）；TU Delft（代尔夫特理工大学）
There is a growing demand for mobile robots to operate in more variable environments, where guaranteeing safe robot navigation is a priority, in addition to time performance. To achieve this, current solutions for local planning use a specific configuration tuned to the characteristics of the application environment. In this paper, we present an approach for developing quality models that can be used by a self-adaptation framework to adapt the local planner configuration at run-time based on the perceived environment. We contribute a definition of a safety model that predicts the safety of a n…
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）
The paper focuses on the development of an autonomous disinfection robot UltraBot to reduce COVID-19 transmission along with other harmful bacteria and viruses. The motivation behind the research is to develop such a robot that is capable of performing disinfection tasks without the use of harmful sprays and chemicals that can leave residues and require airing the room afterward for a long time. UltraBot technology has the potential to offer the most optimal autonomous disinfection performance along with taking care of people, keeping them from getting under the UV-C radiation. The paper highl…
Visually impaired people usually find it hard to travel independently in many public places such as airports and shopping malls due to the problems of obstacle avoidance and guidance to the desired location. Therefore, in the highly dynamic indoor environment, how to improve indoor navigation robot localization and navigation accuracy so that they guide the visually impaired well becomes a problem. One way is to use visual SLAM. However, typical visual SLAM either assumes a static environment, which may lead to less accurate results in dynamic environments or assumes that the targets are all d…
State Key Laboratory of Nuclear Power Safety Monitoring Technology and Equipment, China Nuclear Power Engineering Co.,Ltd（国家核电安全监控技术与装备重点实验室，中国核电工程有限公司）；School of Automation Science and Engineering, South China University of Technology（华南理工大学自动化科学与工程学院）；South-tech Information Technology Co.,LTD（南方科技信息技术有限公司）
Visible light positioning (VLP) is widely believed to be a cost-effective answer to the growing demanded for robot indoor positioning. Considering that some extreme environments require robot to be equipped with a precise and radiation-resistance indoor positioning system for doing difficult work, a novel VLP system with high accuracy is proposed to realize the long-playing inspection and intervention under radiation environment. The proposed system with sufficient radiation-tolerance is critical for operational inspection, maintenance and intervention tasks in nuclear facilities. Firstly, we …
University of Bristol（布里斯托大学）；The Hong Kong University of Science and Technology（香港科技大学）；Shandong University（山东大学）；Jilin University（吉林大学）；Shenzhen Sanhang Industrial Technology Research Institute（深圳三航工业技术研究院）
Mobile robots are becoming increasingly important both for individuals and industries. Mobile robotic technology is not only utilised by experts in this field but is also very popular among amateurs. However, implementing a mobile robot to perform tasks autonomously can be expensive because of the need for various types of sensors and the high price of robot platforms. Hence, in this paper we present a mobile robot localisation and navigation system which uses a LEGO ultrasonic sensor in an indoor map based on the LEGO MINDSTORM NXT. This provides an affordable and ready-to-use option for most…
University of Edinburgh（爱丁堡大学）；University of Liverpool（利物浦大学）；Monash University（莫纳什大学）；University of Oxford（牛津大学）
This paper presents a multimodal indoor odometry dataset, OdomBeyondVision, featuring multiple sensors across the different spectrum and collected with different mobile platforms. Not only does OdomBeyondVision contain the traditional navigation sensors, sensors such as IMUs, mechanical LiDAR, RGBD camera, it also includes several emerging sensors such as the single-chip mmWave radar, LWIR thermal camera and solid-state LiDAR. With the above sensors on UAV, UGV and handheld platforms, we respectively recorded the multimodal odometry data and their movement trajectories in various indoor scenes…
Purdue University（普渡大学）；Iowa State University（爱荷华州立大学）
The paper discusses how robots enable occupant-safe continuous protection for students when schools reopen. Conventionally, fixed air filters are not used as a key pandemic prevention method for public indoor spaces because they are unable to trap the airborne pathogens in time in the entire room. However, by combining the mobility of a robot with air filtration, the efficacy of cleaning up the air around multiple people is largely increased. A disinfection co-robot prototype is thus developed to provide continuous and occupant-friendly protection to people gathering indoors, specifically for …
We propose a framework that allows a mobile robot to build a map of an indoor scenario, identifying and highlighting objects that may be considered a hindrance to people with limited mobility. The map is built by combining recent developments in monocular SLAM with information from inertial sensors of the robot platform, resulting in a metric point cloud that can be further processed to obtain a mesh. The images from the monocular camera are simultaneously analyzed with an object recognition neural network, tuned to detect a particular class of targets. This information is then processed and i…
Indoor Positioning Based on Active Radar Sensing and Passive Reflectors: Reflector Placement Optimization
Institute of Signal Processing and System Theory, University of Stuttgart, Stuttgart, Germany（信号处理与系统理论研究所，斯图加特大学，斯图加特，德国）；Munich Research Center, Huawei Technologies Duesseldorf GmbH, Munich, Germany（慕尼黑研究中心，华为技术杜塞尔多夫有限公司，慕尼黑，德国）
Recognition of occluded objects in unseen indoor environments is a challenging problem for mobile robots. This work proposes a new slicing-based topological descriptor that captures the 3D shape of object point clouds to address this challenge. It yields similarities between the descriptors of the occluded and the corresponding unoccluded objects, enabling object unity-based recognition using a library of trained models. The descriptor is obtained by partitioning an object's point cloud into multiple 2D slices and constructing filtrations (nested sequences of simplicial complexes) on the slice…
Timberline High School（廷伯莱恩高中）；Boise State University（博伊西州立大学）
Localization in indoor environments is essential to further support automation in a wide array of scenarios. Moreover, direction-of-arrival knowledge is essential to supporting high speed millimeter-wave (mmWave) links in indoor environments, since most mmWave links are of a line-of-sight nature to combat the high pathloss in this band. Accurate wireless localization in indoor environments, however, has proved a challenging task due to multi-path fading. Additionally, due to the effects of multi-path fading, methods such as trilateration alone do not result in accurate localization. As such, i…
In this paper, we propose an efficient semantic segmentation framework for indoor scenes, tailored to the application on a mobile robot. Semantic segmentation can help robots to gain a reasonable understanding of their environment, but to reach this goal, the algorithms not only need to be accurate, but also fast and robust. Therefore, we developed an optimized 3D point cloud processing framework based on a Randomized Decision Forest, achieving competitive results at sufficiently high frame rates. We evaluate the capabilities of our method on the popular NYU depth dataset and our own data and …
Ground Technology Research Institute, Agency for Defense Development（国防发展局地面技术研究所）；DGIST（大邱庆北科学技术院）
Currently, mobile robots are developing rapidly and are finding numerous applications in the industry. However, several problems remain related to their practical use, such as the need for expensive hardware and high power consumption levels. In this study, we build a low-cost indoor mobile robot platform that does not include a LiDAR or a GPU. Then, we design an autonomous navigation architecture that guarantees real-time performance on our platform with an RGB-D camera and a low-end off-the-shelf single board computer. The overall system includes SLAM, global path planning, ground segmentati…
In recent years, the mobile robot has been the concern of numerous researcher since they are widely applied in various fields of daily life. This paper applies a virtual robot operating system (ROS) platform to develop a localization system for robot motion. The proposed system is based on the combination of relative and absolute measurement methods, in which the data from the encoder, digital compass, and laser scanner sensor are fused using the extended Kalman filter (EKF). The system also successfully eliminates the errors caused by the environment as well as the error accumulation. The exp…
Oviedo University（奥维耶多大学）；University of Essex（埃塞克斯大学）
Current progresses in home automation and service robotic environment have highlighted the need to develop interoperability mechanisms that allow a standard communication between the two systems. During the development of the DHCompliant protocol, the problem of locating mobile devices in an indoor environment has been investigated. The communication of the device with the location service has been carried out to study the time delay that web services offer in front of the sockets. The importance of obtaining data from real-time location systems portends that a basic tool for interoperability,…
The Chinese University of Hong Kong（香港中文大学）；The University of British Columbia（不列颠哥伦比亚大学）；National Institute of Advanced Industrial Science and Technology（产业技术综合研究所）
Robots are increasingly operating in indoor environments designed for and shared with people. However, robots working safely and autonomously in uneven and unstructured environments still face great challenges. Many modern indoor environments are designed with wheelchair accessibility in mind. This presents an opportunity for wheeled robots to navigate through sloped areas while avoiding staircases. In this paper, we present an integrated software and hardware system for autonomous mobile robot navigation in uneven and unstructured indoor environments. This modular and reusable software framew…
Georgia Institute of Technology（佐治亚理工学院）；Hello Robot Inc.（Hello Robot公司）
Mobile manipulators for indoor human environments can serve as versatile devices that perform a variety of tasks, yet adoption of this technology has been limited. Reducing size, weight, and cost could facilitate adoption, but risks restricting capabilities. We present a novel design that reduces size, weight, and cost, while supporting a variety of tasks. The core design consists of a two-wheeled differential-drive mobile base, a lift, and a telescoping arm configured to achieve Cartesian motion at the end of the arm. Design extensions include a 1 degree-of-freedom (DOF) wrist to stow a tool,…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；NVIDIA Corp.（英伟达公司）
Motion capture is a widely-used technology in robotics research thanks to its precise posi tional measurements with real-time performance. This paper presents ViconMAVLink, a cross-platform open-source software tool that provides indoor positioning services to networked robots. ViconMAVLink converts Vicon motion capture data into proper pose and motion data formats and send localization information to robots using the MAVLink protocol. The software is a convenient tool for mobile robotics researchers to conduct experiments in a controlled indoor environment.
Scene text is an important feature to be extracted, especially in vision-based mobile robot navigation as many potential landmarks such as nameplates and information signs contain text. In this paper, a novel two-step text localization method for Indoor Mobile Robot Navigation is introduced. This method is based on morphological operators and machine learning techniques and can be used in real time environments. The proposed method has two steps. At First, a new set of morphological operators is applied with a particular sequence to extract high contrast areas that have high probability of tex…
Language-Enhanced Mobile Manipulation for Efficient Object Search in Indoor Environments
School of Computation, Information and Technology (CIT), Technical University of Munich（计算、信息与技术学院（CIT），慕尼黑技术大学）；State Key Laboratory for Novel Software Technology and the School of Science and Technology, Nanjing University (Suzhou Campus)（新型软件技术国家重点实验室和南京大学科学技术学院（苏州校区））
Mobile robot navigation in total or partially unknown environments is still an open problem. The path planning algorithms lack completeness and/or performance. Thus, there is the need for complete (i.e., the algorithm determines in finite time either a solution or correctly reports that there is none) and performance (i.e., with low computational complexity) oriented algorithms which need to perform efficiently in real scenarios. In this paper we evaluate the efficiency of two versions of the A star algorithm for mobile robot navigation inside indoor environments with the help of two software …
In this paper we propose a number of tested ways in which a low-budget demo car could be made to navigate an indoor environment. Canny Edge Detection, Supervised Floor Detection and Imitation Learning were used separately and are contrasted in their effectiveness. The equipment used in this paper approximated an autonomous robot configured to work with a mobile device for image processing. This paper does not provide definitive solutions and simply illustrates the approaches taken to successfully achieve autonomous navigation of indoor environments. The successes and failures of all approaches…
University of Antwerp（安特卫普大学）；Flanders Make Strategic Research Centre（法兰德斯制造战略研究中心）
Navigation in varied and dynamic indoor environments remains a complex task for autonomous mobile platforms. Especially when conditions worsen, typical sensor modalities may fail to operate optimally and subsequently provide inapt input for safe navigation control. In this study, we present an approach for the navigation of a dynamic indoor environment with a mobile platform with a single or several sonar sensors using a layered control system. These sensors can operate in conditions such as rain, fog, dust, or dirt. The different control layers, such as collision avoidance and corridor follow…
Majority of the existing robot navigation systems, which facilitate the use of laser range finders, sonar sensors or artificial landmarks, has the ability to locate itself in an unknown environment and then build a map of the corresponding environment. Stereo vision, while still being a rapidly developing technique in the field of autonomous mobile robots, are currently less preferable due to its high implementation cost. This paper aims at describing an experimental approach for the building of a stereo vision system that helps the robots to avoid obstacles and navigate through indoor environ…
A key proficiency an autonomous mobile robot must have to perform high-level tasks is a strong understanding of its environment. This involves information about what types of objects are present, where they are, what their spatial extend is, and how they can be reached, i.e., information about free space is also crucial. Semantic maps are a powerful instrument providing such information. However, applying semantic segmentation and building 3D maps with high spatial resolution is challenging given limited resources on mobile robots. In this paper, we incorporate semantic information into effici…
