This paper contributes a novel and modularized learning-based method for aerial robots navigating cluttered environments containing hard-to-perceive thin obstacles without assuming access to a map or the full pose estimation of the robot. The proposed solution builds upon a semantically-enhanced Variational Autoencoder that is trained with both real-world and simulated depth images to compress the input data, while preserving semantically-labeled thin obstacles and handling invalid pixels in the depth sensor's output. This compressed representation, in addition to the robot's partial state involving its linear/angular velocities and its attitude are then utilized to train an uncertainty-aware 3D Collision Prediction Network in simulation to predict collision scores for candidate action sequences in a predefined motion primitives library. A set of simulation and experimental studies in cluttered environments with various sizes and types of obstacles, including multiple hard-to-perceive thin objects, were conducted to evaluate the performance of the proposed method and compare against an end-to-end trained baseline. The results demonstrate the benefits of the proposed semantically-enhanced deep collision prediction for learning-based autonomous navigation.
School of Automation Science and Engineering, South China University of Technology(华南理工大学自动化科学与工程学院);State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences(中国科学院沈阳自动化研究所机器人重点实验室);State Key Laboratory of Robotics and System, School of Mechanical Engineering and Automation, Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳)机械工程与自动化学院机器人与系统重点实验室)
Vision-Language Navigation for Aerial Robots: Towards the Era of Large Language Models
Defense Innovation Institute, Chinese Academy of Military Sciences(军事科学院国防创新研究院);Intelligent Game and Decision Laboratory(智能博弈与决策实验室);Aerospace Information Research Institute, Chinese Academy of Sciences(中国科学院空天信息创新研究院)
ImagineUAV: Aerial Vision-Language Navigation via World-Action Modeling and Kinodynamic Planning
Pengcheng Laboratory(鹏城实验室);School of Computer Science and Cyber Engineering(计算机科学与网络工程学院);Guangzhou University(广州大学);Southern University of Science and Technology(南方科技大学)
Deep Learning-based Robust Autonomous Navigation of Aerial Robots in Dense Forests
Department of Remote Sensing and Photogrammetry, Finnish Geospatial Research Institute FGI, The National Land Survey of Finland(遥感与摄影测量系,芬兰地理空间研究 institute FGI,芬兰国家土地测绘局)
ViVa-SAFELAND: a New Freeware for Safe Validation of Vision-based Navigation in Aerial Vehicles
Center for Research in Mathematics CIMAT(数学研究中心(CIMAT));Centro de Investigación y de Estudios Avanzados del IPN, CINVESTAV-IPN(墨西哥理工学院高级研究中心(CINVESTAV-IPN))
OnFly: Onboard Zero-Shot Aerial Vision-Language Navigation toward Safety and Efficiency
School of Artificial Intelligence, Sun Yat-Sen University(中山大学人工智能学院);Southern University of Science and Technology(南方科技大学);The Hong Kong University of Science and Technology(香港科技大学)
Fly0: Persistent Metric Anchoring for Zero-Shot Aerial Vision-Language Navigation
National Key Laboratory of Big Data and Decision(大数据与决策国家重点实验室);National University of Defense Technology(国防科技大学);State Key Laboratory of Digital Intelligent Modeling and Simulation(数字智能建模与仿真国家重点实验室);Information Support Force Engineering University(信息支援力量工程大学)
Openfly: A comprehensive platform for aerial vision-language navigation
Shanghai AI Laboratory(上海人工智能实验室);Northwestern Polytechnical University(西北工业大学);Beihang University(北航);Shanghai Jiao Tong University(上海交通大学);The University of Hong Kong(香港大学);Zhejiang University(浙江大学);University of Science and Technology of China(中国科学技术大学);East China University of Science and Technology(东华大学);Fudan University(复旦大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);TeleAI
School of Electrical Engineering and Computer Science, The University of Queensland(昆士兰大学电气工程与计算机科学学院);CSIRO Robotics, Data61(联邦科学与工业研究组织机器人学与Data61研究院)
RINGO: Real-time Navigation with a Guiding Trajectory for Aerial Manipulators in Unknown Environments
Institute of Robotics and Automatic Information System(机器人与自动信息系统研究所);College of Artificial Intelligence(人工智能学院);Tianjin Key Laboratory of Intelligent Robotics(天津市智能机器人重点实验室);Nankai University(南开大学);Institute of Intelligence Technology and Robotic Systems(智能技术与机器人系统研究所);Shenzhen Research Institute of Nankai University(南开大学深圳研究院)
GeoNav: Empowering MLLMs with dual-scale geospatial reasoning for language-goal aerial navigation
College of Systems Engineering, National University of Defense Technology(系统工程学院,国防科技大学);State Key Laboratory of Digital Intelligent Modeling and Simulation(数字智能建模与仿真国家重点实验室);BNRist, Tsinghua University(清华大学BNRist)
Agile in the Face of Delay: Asynchronous End-to-End Learning for Real-World Aerial Navigation
School of Intelligence and Engineering(智能与工程学院);Guangdong Provincial Key Laboratory of Intelligent Morphing Mechanisms and Adaptive Robotics(广东省智能变形机制与自适应机器人重点实验室);Harbin Institute of Technology(哈尔滨工业大学)
Vision-based Navigation of Unmanned Aerial Vehicles in Orchards: An Imitation Learning Approach
Department of Biological and Agricultural Engineering, University of California, Davis, Davis, CA, USA(生物与农业工程系,加州大学戴维斯分校);Department of Mechanical and Aerospace Engineering, University of California, Davis, Davis, CA, USA(机械与航空航天工程系,加州大学戴维斯分校)
Mapless Navigation of a Hybrid Aerial Underwater Vehicle with Deep Reinforcement Learning Through Environmental Generalization
Universidad Tecnológica del Uruguay (UTEC)(乌拉圭技术大学);Centro de Ciências Computacionais (C3) of Universidade Federal do Rio Grande (FURG)(里奥格兰德联邦大学计算科学中心);Universidade Federal de Santa Maria (UFSM)(圣玛丽亚联邦大学)