FINE：面向数据高效视觉语言导航的未来信息导航编码
FINE: Future-Informed Navigation Encoding for Data-Efficient Vision-Language Navigation
AI总结：
FINE通过从现有演示中提取未来地标信息作为辅助监督，提升视觉语言导航策略的数据效率，在有限演示下显著提高成功率。
AI中文摘要：
将视觉语言导航（VLN）策略适应新环境成本高昂，因为每条额外路线和指令都需要具身演示。然而，标准的观察-动作训练仅利用了每条轨迹中已包含信息的一小部分。特别是，未来观察揭示了智能体将遇到的与指令相关的地标，包括它们的外观及其在3D空间中的排列方式。我们提出了FINE，一种未来信息导航编码框架，从现有演示中提取这种潜在监督。FINE为VLN骨干网络配备了两种互补的辅助表示。首先，显式地标标记遵循指令指定的有序地标，并被训练以在语义2D补丁特征空间和视点相关的3D几何特征空间中预测未来地标区域。其次，隐式未来标记学习区分实际到达的地标状态与由视频世界模型生成的合理同场景反事实未来。在R2R-CE和RxR-CE的val-unseen上，FINE在完整训练数据下分别将InternVLA-N1的成功率提高了2.6和4.5个百分点。更重要的是，当演示变得有限时，收益增加：在70%演示预算下，FINE将成功率提高了6.8个百分点，恢复了因减少训练演示而损失性能的大约三分之一。项目页面可在该https URL获取。
英文摘要：
Adapting vision-language navigation (VLN) policies to new environments is expensive because every additional route and instruction requires an embodied demonstration. Yet standard observation-to-action training uses only a small fraction of the information already contained in each trajectory. In particular, future observations reveal the instruction-relevant landmarks that the agent will encounter, including what they look like and how they are arranged in 3D. We introduce FINE, a Future-Informed Navigation Encoding framework that extracts this latent supervision from existing demonstrations. FINE equips a VLN backbone with two complementary auxiliary representations. First, explicit landmark tokens follow the ordered landmarks specified by the instruction and are trained to predict future landmark regions in both semantic 2D patch-feature space and viewpoint-dependent 3D geometric feature space. Second, an implicit future token learns to distinguish the landmark state that is actually reached from plausible same-scene counterfactual futures generated by a video world model. On R2R-CE and RxR-CE val-unseen, FINE improves InternVLA-N1 by 2.6 and 4.5 success-rate points, respectively, at full training data. More importantly, as demonstrations become limited, the benefit grows: at a 70% demonstration budget, FINE improves success rate by 6.8 points, recovering roughly one-third of the performance lost by reducing the training demonstrations. Project page is available at https://finevln.github.io/.
