How can we build robots for open-world semantic navigation tasks, like searching for target objects in novel scenes? While foundation models have the rich knowledge and generalisation needed for these tasks, a suitable scene representation is needed to connect them into a complete robot system. We address this with Open Scene Graphs (OSGs), a topo-semantic representation that retains and organises open-set scene information for these models, and has a structure that can be configured for different environment types. We integrate foundation models and OSGs into the OpenSearch system for Open World Object-Goal Navigation, which is capable of searching for open-set objects specified in natural language, while generalising zero-shot across diverse environments and embodiments. Our OSGs enhance reasoning with Large Language Models (LLM), enabling robust object-goal navigation outperforming existing LLM approaches. Through simulation and real-world experiments, we validate OpenSearch's generalisation across varied environments, robots and novel instructions.
HyperDCM: Dynamic Cluster Memory Replay in Hyperbolic Space for Continual Robotic Navigation Across Scenes
Software Engineering Institute, East China Normal University(华东师范大学软件工程学院);School of Geospatial Information, Information Engineering University(信息工程大学地理空间信息学院);University of Shanghai for Science and Technology(上海理工大学);Shanghai Jiao Tong University(上海交通大学)
NavTuner: Learning a Scene-Sensitive Family of Navigation Policies
College of Computing, Georgia Institute of Technology(佐治亚理工学院计算学院);Institute for Robotics and Intelligent Machines, Georgia Institute of Technology(佐治亚理工学院机器人与智能机器研究所)
Learning Actions for Drift-Free Navigation in Highly Dynamic Scenes
IIIT-Hyderabad(国际信息技术研究所海得拉巴分校);The University of Texas at Austin(德克萨斯大学奥斯汀分校)
我们着手一个迄今未报告的问题,自主机器人(自动驾驶汽车)在动态场景中导航的方式,减少其定位误差和最终的累积漂移或绝对轨迹误差,这是在这样的动态场景中明显。以非常受欢迎的Velodyne-16 3D LIDAR作为主要的传感模式,以及精确的基于LIDAR的定位和映射算法LOAM作为状态估计框架,我们表明,在没有导航策略的情况下,漂移在移动物体的存在下迅速积累。为了克服这一点,我们通过一组合适的奖励和惩罚函数来学习导致漂移最小化导航的动作。我们使用近端策略优化,一类深度强化学习方法,来学习导致漂移最小化轨迹的动作。我们通过对各种合成的,但照片般逼真的场景通过CARLA模拟器提供的优越性能的建议框架相对于不采用这种政策的方法进行了广泛的比较。
Learning Robot Visual Navigation in Crowds via Intention-Aware Scene Representations
Shenzhen Key Laboratory of Robotics Perception and Intelligence, Department of Electronic and Electrical Engineering, SUSTech(南方科技大学电子与电气工程系深圳市机器人感知与智能重点实验室);Jiaxing Research Institute, SUSTech(南方科技大学嘉兴研究院);Research Institute of MA&EI, Peng Cheng Laboratory(鹏城实验室机器人与人工智能研究所)
Where to Fetch: Extracting Visual Scene Representation from Large Pre-Trained Models for Robotic Goal Navigation
School of Control Science and Engineering, Shandong University(山东大学控制科学与工程学院);Campus Security Smart Brain Shandong Engineering Research Center(山东校园安全智能脑工程研究中心)
Towards Target-Driven Visual Navigation in Indoor Scenes via Generative Imitation Learning
Nanjing University of Aeronautics and Astronautics(南京航空航天大学);National University of Defense Technology(国防科技大学);University of Maryland, College Park(马里兰大学帕克分校)
GUIDE: Goal-Initialized Directional Understanding for End-to-End Legged Navigation
Institute of Cyber-Systems and Control, Zhejiang University(浙江大学控制科学与工程学院);Institute of Artificial Intelligence (TeleAI), China Telecom(中国电信人工智能研究院(TeleAI));Oxford Robotics Institute, University of Oxford(牛津大学牛津机器人研究所);Center for Robotics, University of Bonn(波恩大学机器人中心);Department of Mechanical Engineering, Massachusetts Institute of Technology(麻省理工学院机械工程系)
BiliVLA: Scene-Aware Vision-Language-Action Model with Reinforcement Learning for Autonomous Biliary Endoscopic Navigation
The Chinese University of Hong Kong(香港中文大学);The Third Affiliated Hospital of Sun Yat-sen University(中山大学附属第三医院);University of Cambridge(剑桥大学);University of California, Davis(加州大学戴维斯分校)
Immersive Explainability: Visualizing Robot Navigation Decisions through XAI Semantic Scene Projections in Virtual Reality
University of Bonn(波恩大学);Lamarr Institute for Machine Learning and Artificial Intelligence(拉马尔人工智能与机器学习研究所);Center for Robotics(机器人中心);University of Mainz(美因茨大学);Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS(弗劳恩霍夫智能分析与信息系统研究所)
NavIsaacLab: Generating Realistic Crowd via Parallel Robot Learning for Benchmarking Human-aware Navigation
Southern University of Science and Technology(南方科技大学);National University of Singapore(新加坡国立大学);Peng Cheng Laboratory(鹏城实验室);Jiaxing Research Institute, Southern University of Science and Technology(南方科技大学嘉兴研究院)
Reinforcement Learning-based Visual Navigation with Information-Theoretic Regularization
Nanjing University of Aeronautics and Astronautics(南京航空航天大学);National University of Defense Technology(国防科技大学);Zhengzhou University(郑州大学);University of Maryland, College Park(马里兰大学帕克分校)
Open Scene Graphs for Open-World Object-Goal Navigation
Smart Systems Institute, National University of Singapore(新加坡国立大学智能系统研究所);College of Computing and Information Science, Cornell University(康奈尔大学计算机与信息科学学院)