arXiv 2411.160532025-03-16cs.CVcs.AI

UnitedVLN：用于连续视觉-语言导航的可泛化高斯泼溅

UnitedVLN: Generalizable Gaussian Splatting for Continuous Vision-Language Navigation

Nanjing University of Science and Technology（南京理工大学）；Northwest Polytechnical University（西北工业大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Tsinghua University（清华大学）

Guangzhao Dai, Jian Zhao, Yuantao Chen, Yusen Qin, Hao Zhao, Guosen Xie, Yazhou Yao, Xiangbo Shu, Xuelong Li

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对连续视觉-语言导航中的遮挡与观测不足问题，提出基于可泛化3DGS的UnitedVLN预训练范式，联合渲染360°图像与语义特征，并在VLN-CE基准上达到最优。

摘要：

视觉-语言导航（VLN）是指智能体遵循指令到达目标目的地，近年来已取得显著进展。与在具有预定义轨迹的离散环境中导航不同，连续环境中的VLN（VLN-CE）带来了更大挑战，因为智能体可以自由导航至任何无遮挡位置，并且更容易受到视觉遮挡或盲点的影响。近期方法试图通过想象未来环境来解决这一问题，即借助预测的未来视觉图像或语义特征，而非仅依赖当前观测。然而，这些基于RGB和基于特征的方法缺乏对有效导航至关重要的直观外观级信息或高级语义复杂性。为克服这些局限，我们提出一种新颖的、可泛化的、基于3DGS的预训练范式，称为UnitedVLN，它通过联合渲染高保真360视觉图像和语义特征，使智能体能够更好地探索未来环境。UnitedVLN采用两个关键方案：先搜索后查询采样，以及先分离后联合渲染；它们促进了对神经基元的高效利用，有助于整合外观与语义信息，从而实现更稳健的导航。大量实验表明，UnitedVLN在现有VLN-CE基准上优于最先进方法。

英文摘要：

Vision-and-Language Navigation (VLN), where an agent follows instructions to reach a target destination, has recently seen significant advancements. In contrast to navigation in discrete environments with predefined trajectories, VLN in Continuous Environments (VLN-CE) presents greater challenges, as the agent is free to navigate any unobstructed location and is more vulnerable to visual occlusions or blind spots. Recent approaches have attempted to address this by imagining future environments, either through predicted future visual images or semantic features, rather than relying solely on current observations. However, these RGB-based and feature-based methods lack intuitive appearance-level information or high-level semantic complexity crucial for effective navigation. To overcome these limitations, we introduce a novel, generalizable 3DGS-based pre-training paradigm, called UnitedVLN, which enables agents to better explore future environments by unitedly rendering high-fidelity 360 visual images and semantic features. UnitedVLN employs two key schemes: search-then-query sampling and separate-then-united rendering, which facilitate efficient exploitation of neural primitives, helping to integrate both appearance and semantic information for more robust navigation. Extensive experiments demonstrate that UnitedVLN outperforms state-of-the-art methods on existing VLN-CE benchmarks.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。