Air-ground robots (AGRs) are widely used in surveillance and disaster response due to their exceptional mobility and versatility (i.e., flying and driving). Current AGR navigation systems perform well in static occlusion-prone environments (e.g., indoors) by using 3D semantic occupancy networks to predict occlusions for complete local mapping and then computing Euclidean Signed Distance Field (ESDF) for path planning. However, these systems face challenges in dynamic, severe occlusion scenes (e.g., crowds) due to limitations in perception networks' low prediction accuracy and path planners' high computation overhead. In this paper, we propose OMEGA, which contains OccMamba with an Efficient AGR-Planner to address the above-mentioned problems. OccMamba adopts a novel architecture that separates semantic and occupancy prediction into independent branches, incorporating two mamba blocks within these branches. These blocks efficiently extract semantic and geometric features in 3D environments with linear complexity, ensuring that the network can learn long-distance dependencies to improve prediction accuracy. Semantic and geometric features are combined within the Bird's Eye View (BEV) space to minimise computational overhead during feature fusion. The resulting semantic occupancy map is then seamlessly integrated into the local map, providing occlusion awareness of the dynamic environment. Our AGR-Planner utilizes this local map and employs kinodynamic A* search and gradient-based trajectory optimization to guarantee planning is ESDF-free and energy-efficient. Extensive experiments demonstrate that OccMamba outperforms the state-of-the-art 3D semantic occupancy network with 25.0% mIoU. End-to-end navigation experiments in dynamic scenes verify OMEGA's efficiency, achieving a 96% average planning success rate. Code and video are available at https://jmwang0117.github.io/OMEGA/.
Aalto University（阿尔托大学）；Tampere Univ. of Tech.（坦佩雷理工大学）
This paper presents a novel method for visual-inertial odometry. The method is based on an information fusion framework employing low-cost IMU sensors and the monocular camera in a standard smartphone. We formulate a sequential inference scheme, where the IMU drives the dynamical model and the camera frames are used in coupling trailing sequences of augmented poses. The novelty in the model is in taking into account all the cross-terms in the updates, thus propagating the inter-connected uncertainties throughout the model. Stronger coupling between the inertial and visual data sources leads to…
Technical University Munich（慕尼黑工业大学）；M2C ExpertControl GmbH（M2C ExpertControl公司）；SBB AG（瑞士联邦铁路公司）
We present an approach for optical navigation in unstructured, dynamic railroad environments. We propose a way how to cope with the estimation of the train motion from sole observations of the planar track bed. The occasional significant occlusions during the operation of the train limit the available observation to this difficult to track, repetitive area. This approach is a step towards replacement of the expensive train management infrastructure with local intelligence on the train for SmartRail 4.0. We derive our approach for robust estimation of translation and rotation in this difficult …
We introduce a framework for navigating through cluttered environments by connecting multiple cameras together while simultaneously preserving privacy. Occlusions and obstacles in large environments are often challenging situations for navigation agents because the environment is not fully observable from a single camera view. Given multiple camera views of an environment, our approach learns to produce a multiview scene representation that can only be used for navigation, provably preventing one party from inferring anything beyond the output task. On a new navigation dataset that we will pub…
Zhejiang University（浙江大学）；Huzhou Institute of Zhejiang University（浙江大学湖州研究院）
In autonomous navigation of mobile robots, sensors suffer from massive occlusion in cluttered environments, leaving significant amount of space unknown during planning. In practice, treating the unknown space in optimistic or pessimistic ways both set limitations on planning performance, thus aggressiveness and safety cannot be satisfied at the same time. However, humans can infer the exact shape of the obstacles from only partial observation and generate non-conservative trajectories that avoid possible collisions in occluded space. Mimicking human behavior, in this paper, we propose a method…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Despite the rapid advancement of navigation algorithms, mobile robots often produce anomalous behaviors that can lead to navigation failures. The ability to detect such anomalous behaviors is a key component in modern robots to achieve high-levels of autonomy. Reactive anomaly detection methods identify anomalous task executions based on the current robot state and thus lack the ability to alert the robot before an actual failure occurs. Such an alert delay is undesirable due to the potential damage to both the robot and the surrounding objects. We propose a proactive anomaly detection network…
Overcoming the Fear of the Dark: Occlusion-Aware Model-Predictive Planning for Automated Vehicles Using Risk Fields
Netherlands Organisation for Applied Scientific Research（荷兰应用科学研究组织）；Eindhoven University of Technology（埃因霍温理工大学）；KTH Royal Institute of Technology（KTH皇家理工学院）；Scania CV AB（斯堪尼亚商用车有限公司）
Active surgical robots lack acceptance in clinical practice, because they do not offer the flexibility and usability required for a versatile usage: the systems require a large installation space or a complicated registration step, where the preoperative plan is aligned to the patient and transformed to the base frame of the robot. In this paper, a navigation system for robotic osteotomies is designed, which uses the raw depth images from a camera mounted on the flange of a lightweight robot arm. Consequently, the system does not require any rigid attachment of the robot or fiducials to the bo…
DualVision ArthroNav: Investigating Opportunities to Enhance Localization and Reconstruction in Image-based Arthroscopy Navigation via External Cameras
University of Arkansas（阿肯色大学）；Arthrex Inc.（阿瑟克斯公司）
MolmoSpaces: A Large-Scale Open Ecosystem for Robot Navigation and Manipulation
Allen Institute for AI（艾伦人工智能研究所）；University of Washington（华盛顿大学）；University of California, Los Angeles（加州大学洛杉矶分校）；University of California, Berkeley（加州大学伯克利分校）
When performing visual servoing or object tracking tasks, active sensor planning is essential to keep targets in sight or to relocate them when missing. In particular, when dealing with a known target missing from the sensor's field of view, we propose using prior knowledge related to contextual information to estimate its possible location. To this end, this study proposes a Dynamic Bayesian Network that uses contextual information to effectively search for targets. Monte Carlo particle filtering is employed to approximate the posterior probability of the target's state, from which uncertaint…
Balgrist University Hospital, University of Zurich（苏黎世大学巴尔格利斯特大学医院）；ETH Zurich（苏黎世联邦理工学院）；OR-X Translational Center for Surgery, Balgrist University Hospital, University of Zurich（苏黎世大学巴尔格利斯特大学医院OR-X外科转化中心）；Technical University Munich（慕尼黑工业大学）
Dual-BEV Nav: Dual-layer BEV-based Heuristic Path Planning for Robotic Navigation in Unstructured Outdoor Environments
Software Engineering Institute, East China Normal University（华东师范大学软件工程学院）；School of Geospatial Information, Information Engineering University（信息工程大学地理空间信息学院）；College of Computer and Cyber Security, Fujian Normal University（福建师范大学计算机与网络空间安全学院）
针对非结构化室外环境路径规划难题，提出 Dual-BEV Nav 方法，通过整合局部与全局 BEV 可通行性建立双层启发式规划范式，显著提升了时间距离预测精度并实现了长距离室外导航。
Schrödinger's Navigator: Imagining an Ensemble of Futures for Zero-Shot Object Navigation
Fudan University（复旦大学）；Shanghai Jiao Tong University（上海交通大学）；Shanghai University of International Business and Economics（上海对外经贸大学）；Shanghai Innovation Institute（上海创新研究院）
Towards Real-Time Autonomous Navigation: Transformer-Based Catheter Tip Tracking in Fluoroscopy
Surgical & Interventional Engineering School of Biomedical Engineering & Imaging Sciences Kings College London（生物医学工程与成像科学学院手术与介入工程系伦敦国王学院）；School of Engineering & Materials Science Queen Mary London（工程与材料科学学院女王玛丽学院伦敦）
Allen Institute for AI（艾伦人工智能研究所）；University of Washington（华盛顿大学）
The domain of Embodied AI has recently witnessed substantial progress, particularly in navigating agents within their environments. These early successes have laid the building blocks for the community to tackle tasks that require agents to actively interact with objects in their environment. Object manipulation is an established research domain within the robotics community and poses several challenges including manipulator motion, grasping and long-horizon planning, particularly when dealing with oft-overlooked practical setups involving visually rich and complex scenes, manipulation using m…
Unified Linear Parametric Map Modeling and Perception-aware Trajectory Planning for Mobile Robotics
School of Information Science and Engineering, Shenyang University of Technology（沈阳理工大学信息科学与工程学院）；State Key Laboratory of Synthetic Automation for Process Industries, Northeastern University（东北大学过程工业综合自动化重点实验室）；State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所机器人重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）
Autonomous robotic systems operating in human environments must understand their surroundings to make accurate and safe decisions. In crowded human scenes with close-up human-robot interaction and robot navigation, a deep understanding requires reasoning about human motion and body dynamics over time with human body pose estimation and tracking. However, existing datasets either do not provide pose annotations or include scene types unrelated to robotic applications. Many datasets also lack the diversity of poses and occlusions found in crowded human scenes. To address this limitation we intro…
CMRL: Collision-Aware and Memory-Enhanced Reinforcement Learning for UAV Navigation in Multi-Scale Obstacle Environments
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Guangdong Key Laboratory of Big Data Analysis and Processing（广东省大数据分析与处理重点实验室）
University of Oxford（牛津大学）；Oxford Robotics Institute（牛津机器人研究所）
Visual motion estimation is an integral and well-studied challenge in autonomous navigation. Recent work has focused on addressing multimotion estimation, which is especially challenging in highly dynamic environments. Such environments not only comprise multiple, complex motions but also tend to exhibit significant occlusion. Previous work in object tracking focuses on maintaining the integrity of object tracks but usually relies on specific appearance-based descriptors or constrained motion models. These approaches are very effective in specific applications but do not generalize to the full…
Conventional computer-assisted orthopaedic navigation systems rely on the tracking of dedicated optical markers for patient poses, which makes the surgical workflow more invasive, tedious, and expensive. Visual tracking has recently been proposed to measure the target anatomy in a markerless and effortless way, but the existing methods fail under real-world occlusion caused by intraoperative interventions. Furthermore, such methods are hardware-specific and not accurate enough for surgical applications. In this paper, we propose a RGB-D sensing-based markerless tracking method that is robust a…
Lab for Autonomous Robotics Research (LARR), Seoul National University（首尔国立大学自主机器人研究实验室（LARR））；Computer Graphics and Vision Lab (GrUVi), Simon Fraser University（西蒙弗雷泽大学计算机图形与视觉实验室（GrUVi））
Detection of moving objects is an essential capability in dealing with dynamic environments. Most moving object detection algorithms have been designed for color images without depth. For robotic navigation where real-time RGB-D data is often readily available, utilization of the depth information would be beneficial for obstacle recognition. Here, we propose a simple moving object detection algorithm that uses RGB-D images. The proposed algorithm does not require estimating a background model. Instead, it uses an occlusion model which enables us to estimate the camera pose on a background con…
