Goal2Pixel: 将目标锚定到像素以实现视觉语言导航
Goal2Pixel: Grounding Goals to Pixels for Vision-Language Navigation
AI总结：
提出Goal2Pixel范式，通过将连续环境中的视觉语言导航（VLN-CE）重新定义为可导航像素锚定，利用图像平面作为统一空间接口，预测可见导航像素并反投影为3D航点，结合可见性感知关键帧记忆和坐标感知辅助损失，在减少VLM调用次数的同时实现竞争性性能。
摘要：
视觉语言模型（VLM）已成为连续环境中视觉语言导航（VLN-CE）的常见基础。然而，大多数基于VLM的方法将导航视为低级动作预测，这种接口模糊、受限于短视运动基元，且由于重复的VLM查询而效率低下。我们提出Goal2Pixel，一种纯基于像素的范式，将VLN-CE重新定义为可导航像素锚定。Goal2Pixel不预测动作，而是使用图像平面作为VLM推理与机器人运动之间的统一空间接口：模型预测一个对智能体可见的可导航像素，该像素被反投影为3D航点以进行前向导航。对于非前向动作，我们在图像平面上附加辅助指令区域，其中左/右/下区域分别解释为左转、右转和停止。为了实现长程导航，我们提出了一种可见性感知的关键帧记忆，用于紧凑且信息丰富的历史表示。为了将预训练的VLM适应于可导航像素锚定，我们引入了语义嵌入和坐标感知辅助损失。Goal2Pixel在需要比先前方法更少的VLM推理调用的情况下，实现了具有竞争力的最新性能。在R2R-CE Val-Unseen上，它以每集仅7.75次VLM调用达到54.1%的SR和52.5%的SPL，而直接动作预测在32.9%的SR下需要46.62次调用，减少了6倍。同样的趋势在RxR-CE上也成立。项目页面：https://baobao0926.github.io/Goal2Pixel/。
英文摘要：
Vision-language models (VLMs) have become a common foundation for vision-and-language navigation in continuous environments (VLN-CE). Yet most VLM-based methods cast navigation as low-level action prediction, an interface that is ambiguous, tied to short-horizon motion primitives, and inefficient due to repeated VLM querying. We propose Goal2Pixel, a pure pixel-based paradigm that reformulates VLN-CE as navigable pixel grounding. Rather than predicting actions, Goal2Pixel uses the image plane as a unified spatial interface between VLM reasoning and robot motion: the model predicts a visible navigable pixel to the agent, which is back-projected into a 3D waypoint for forward navigation. For non-forward actions, we append auxiliary directive regions to the image plane, where the left/right/bottom regions are interpreted as turning left, turning right, and stopping, respectively. To enable long-horizon navigation, we propose a visibility-aware keyframe memory for compact and informative history representation. To adapt pretrained VLMs to navigable pixel grounding, we introduce semantic embeddings and coordinate-aware auxiliary losses. Goal2Pixel achieves competitive state-of-the-art performance while requiring fewer VLM inference calls than prior methods. On R2R-CE Val-Unseen it achieves 54.1% SR and 52.5% SPL with just 7.75 VLM calls per episode, 6x fewer than the 46.62 required by direct action prediction at 32.9% SR. The same trend holds on RxR-CE.Project Page: https://baobao0926.github.io/Goal2Pixel/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
