This paper introduces a novel perception framework that has the ability to identify and track objects in autonomous vehicle's field of view. The proposed algorithms don't require any training for achieving this goal. The framework makes use of ego-vehicle's pose estimation and a KD-Tree-based segmentation algorithm to generate object clusters. In turn, using a VFH technique, the geometry of each identified object cluster is translated into a multi-modal PDF and a motion model is initiated with every new object cluster for the purpose of robust spatio-temporal tracking. The methodology further uses statistical properties of high-dimensional probability density functions and Bayesian motion model estimates to identify and track objects from frame to frame. The effectiveness of the methodology is tested on a KITTI dataset. The results show that the median tracking accuracy is around 91% with an end-to-end computational time of 153 milliseconds
ARIANNA stands for pAth Recognition for Indoor Assisted Navigation with Augmented perception. It is a flexible and low cost navigation system for vi- sually impaired people. Arianna permits to navigate colored paths painted or sticked on the floor revealing their directions through vibrational feedback on commercial smartphones.
SSF-PAN: Semantic Scene Flow-Based Perception for Autonomous Navigation in Traffic Scenarios
Research Institute of Trustworthy Autonomous Systems, Southern University of Science and Technology（南方科技大学可信自主系统研究院）；Shenzhen Deeproute.ai Co.,Ltd（深圳元戎启行科技有限公司）
ST-Booster: An Iterative SpatioTemporal Perception Booster for Vision-and-Language Navigation in Continuous Environments
Robotics and Control Laboratory, the School of Advanced Manufacturing and Robotics, and the State Key Laboratory of Turbulence and Complex Systems, Peking University（机器人与控制实验室、先进制造与机器人学院、湍流与复杂系统国家重点实验室，北京大学）；Defense Innovation Institute, Academy of Military Sciences（国防创新研究院，军事科学院）；Tianjin Artificial Intelligence Innovation Center（天津人工智能创新中心）；Harbin Institute of Technology（哈尔滨工业大学）
Learning visuomotor control policies in robotic systems is a fundamental problem when aiming for long-term behavioral autonomy. Recent supervised-learning-based vision and motion perception systems, however, are often separately built with limited capabilities, while being restricted to few behavioral skills such as passive visual odometry (VO) or mobile robot visual localization. Here we propose an approach to unify those successful robot perception systems for active target-driven navigation tasks via reinforcement learning (RL). Our method temporally incorporates compact motion and visual p…
NUS Graduate School for Integrative Sciences and Engineering（新加坡国立大学综合科学与工程研究生院）
We propose to take a novel approach to robot system design where each building block of a larger system is represented as a differentiable program, i.e. a deep neural network. This representation allows for integrating algorithmic planning and deep learning in a principled manner, and thus combine the benefits of model-free and model-based methods. We apply the proposed approach to a challenging partially observable robot navigation task. The robot must navigate to a goal in a previously unseen 3-D environment without knowing its initial location, and instead relying on a 2-D floor map and vis…
We introduce a framework for navigating through cluttered environments by connecting multiple cameras together while simultaneously preserving privacy. Occlusions and obstacles in large environments are often challenging situations for navigation agents because the environment is not fully observable from a single camera view. Given multiple camera views of an environment, our approach learns to produce a multiview scene representation that can only be used for navigation, provably preventing one party from inferring anything beyond the output task. On a new navigation dataset that we will pub…
Gaussian Mixture-Based Inverse Perception Contract for Uncertainty-Aware Robot Navigation
Department of Computer Science, Columbia University（计算机科学系，哥伦比亚大学）；Department of Computer Science and Engineering, Texas A&M University（计算机科学与工程系，德克萨斯大学）
Jacobs University Bremen（不来梅雅各布大学）；ShanghaiTech University（上海科技大学）
Underwater robot interventions require a high level of safety and reliability. A major challenge to address is a robust and accurate acquisition of localization estimates, as it is a prerequisite to enable more complex tasks, e.g. floating manipulation and mapping. State-of-the-art navigation in commercial operations, such as oil & gas production (OGP), rely on costly instrumentation. These can be partially replaced or assisted by visual navigation methods, especially in deep-sea scenarios where equipment deployment has high costs and risks. Our work presents a multimodal approach that adapts …
Visual monitoring operations underwater require both observing the objects of interest in close-proximity, and tracking the few feature-rich areas necessary for state estimation.This paper introduces the first navigation framework, called AquaVis, that produces on-line visibility-aware motion plans that enable Autonomous Underwater Vehicles (AUVs) to track multiple visual objectives with an arbitrary camera configuration in real-time. Using the proposed pipeline, AUVs can efficiently move in 3D, reach their goals while avoiding obstacles safely, and maximizing the visibility of multiple object…
DeepIPCv2: LiDAR-powered Robust Environmental Perception and Navigational Control for Autonomous Vehicle
Department of Computer Science and Electronics, Universitas Gadjah Mada（计算机科学与电子系，加查马达大学）；Department of Computer Science and Engineering, Toyohashi University of Technology（计算机科学与工程系，toyohashi技术大学）
Integrated YOLOP Perception and Lyapunov-based Control for Autonomous Mobile Robot Navigation on Track
Graduate School of Science and Technology（科学与技术研究生学院）；Master’s Program in Science and Technology（科学技术硕士项目）；Green Science and Engineering Division（绿色科学与工程系）
Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）；Université de Montréal（蒙特利尔大学）
In this work, we consider the problem of learning a perception model for monocular robot navigation using few annotated images. Using a Vision Transformer (ViT) pretrained with a label-free self-supervised method, we successfully train a coarse image segmentation model for the Duckietown environment using 70 training images. Our model performs coarse image segmentation at the 8x8 patch level, and the inference resolution can be adjusted to balance prediction granularity and real-time perception constraints. We study how best to adapt a ViT to our task and environment, and find that some lightw…
Traversability prediction is a fundamental perception capability for autonomous navigation. The diversity of data in different domains imposes significant gaps to the prediction performance of the perception model. In this work, we make efforts to reduce the gaps by proposing a novel coarse-to-fine unsupervised domain adaptation (UDA) model - CALI. Our aim is to transfer the perception model with high data efficiency, eliminate the prohibitively expensive data labeling, and improve the generalization capability during the adaptation from easy-to-obtain source domains to various challenging tar…
Uncertainty-Informed Active Perception for Open Vocabulary Object Goal Navigation
Center for Robotics, University of Bonn（波恩大学机器人中心）；MAVLab, TU Delft（代尔夫特理工大学MAVLab）；Lamarr Institute for Machine Learning and Artificial Intelligence（机器学习与人工智能拉马尔研究所）
Stanford University（斯坦福大学）；Facebook AI Research（脸书人工智能研究院）
A complex visual navigation task puts an agent in different situations which call for a diverse range of visual perception abilities. For example, to "go to the nearest chair", the agent might need to identify a chair in a living room using semantics, follow along a hallway using vanishing point cues, and avoid obstacles using depth. Therefore, utilizing the appropriate visual perception abilities based on a situational understanding of the visual environment can empower these navigation models in unseen visual environments. We propose to train an agent to fuse a large set of visual representa…
GE Research（通用电气研究院）；Motional（Motional公司）；GE Digital（通用电气数字部门）
Off-road autonomous unmanned ground vehicles (UGVs) are being developed for military and commercial use to deliver crucial supplies in remote locations, help with mapping and surveillance, and to assist war-fighters in contested environments. Due to complexity of the off-road environments and variability in terrain, lighting conditions, diurnal and seasonal changes, the models used to perceive the environment must handle a lot of input variability. Current datasets used to train perception models for off-road autonomous navigation lack of diversity in seasons, locations, semantic classes, as w…
University of Washington（华盛顿大学）；Facebook AI Research（脸书人工智能研究院）；Georgia Institute of Technology（佐治亚理工学院）
We propose SplitNet, a method for decoupling visual perception and policy learning. By incorporating auxiliary tasks and selective learning of portions of the model, we explicitly decompose the learning objectives for visual navigation into perceiving the world and acting on that perception. We show dramatic improvements over baseline models on transferring between simulators, an encouraging step towards Sim2Real. Additionally, SplitNet generalizes better to unseen environments from the same simulator and transfers faster and more effectively to novel embodied navigation tasks. Further, given …
FLAP: FOV-Constrained Active Perception Planning for Prior-Map-Free 3D Navigation
The State Key Laboratory of Industrial Control Technology, College of Control Science and Engineering, Zhejiang University（浙江大学控制科学与工程学院工业控制技术国家重点实验室）；Huzhou Institute, Zhejiang University（浙江大学湖州研究院）；Huzhou Key Laboratory of Autonomous System（湖州市自动驾驶系统重点实验室）；Shanghai Institute of Special Equipment Inspection and Technical Research Co., Ltd（上海市特种设备监督检验技术研究院有限公司）
Autonomous Navigation of Cloud-Controlled Quadcopters in Confined Spaces Using Multi-Modal Perception and LLM-Driven High Semantic Reasoning
Department of Mechanical Engineering（机械工程系）；Rajshahi University of Engineering and Technology（拉贾沙希工程与技术大学）；Department of Industrial and Production Engineering（工业与生产工程系）；Shahjalal University of Science and Technology（沙赫jalal科学与技术大学）；Bangladesh University of Engineering and Technology（孟加拉工程与技术大学）；Department of Urban and Regional Planning（城市与区域规划系）；Department of Computer Science Engineering（计算机科学与工程系）；United International University（联合国际大学）
A 2D Image Feedback-Based Autonomous Endovascular Navigation Method: A Perception and Control Framework
Beijing Institute of Technology（北京理工大学）；Advanced Institute for Ocean Research, Southern University of Science and Technology（南方科技大学海洋科学与工程研究院）；Shenzhen Institute of Advanced Biomedical Robot Company Limited（深圳先进生物医学机器人有限公司）；Southern University of Science and Technology（南方科技大学）；School of Mechanical, Electronic and Control Engineering, Beijing Jiaotong University（北京交通大学机械与电子控制工程学院）；Aerospace Center Hospital（航天中心医院）
