弱监督VLM引导的部分对比学习用于视觉语言导航
Weakly-supervised VLM-guided Partial Contrastive Learning for Visual Language Navigation
AI总结：
针对视觉语言导航中动态视角感知和VLM微调成本高的问题，提出弱监督部分对比学习（WPCL），无需微调VLM即可提升智能体识别能力，在多个基准上超越基线。
摘要：
视觉语言导航（VLN）是具身智能领域的一项基础任务，关注智能体基于自然语言指令在复杂环境中导航的能力。尽管现有方法已取得进展，但这些方法常常面临一些共同挑战。首先，它们依赖预训练骨干模型进行视觉感知，而这些模型难以应对VLN场景中的动态视角。其次，使用未经微调的预训练LLM或VLM时，由于缺乏VLN领域知识，性能受到限制。第三，虽然微调LLM和VLM可以改善结果，但其计算成本高于不进行微调的方法。为了解决这些局限，我们提出了弱监督部分对比学习（WPCL），该方法通过有效整合预训练VLM知识到感知过程中，增强智能体在VLN场景中从动态视角识别物体的能力，且无需对VLM进行微调。我们的方法增强了智能体解释和响应环境线索的能力，同时确保了计算效率。实验结果表明，我们的方法在多个基准上优于基线方法，验证了该方法的有效性、鲁棒性和泛化性。
英文摘要：
Visual Language Navigation (VLN) is a fundamental task within the field of Embodied AI, focusing on the ability of agents to navigate complex environments based on natural language instructions. Despite the progress made by existing methods, these methods often present some common challenges. First, they rely on pre-trained backbone models for visual perception, which struggle with the dynamic viewpoints in VLN scenarios. Second, the performance is limited when using pre-trained LLMs or VLMs without fine-tuning, due to the absence of VLN domain knowledge. Third, while fine-tuning LLMs and VLMs can improve results, their computational costs are higher than those without fine-tuning. To address these limitations, we propose Weakly-supervised Partial Contrastive Learning (WPCL), a method that enhances an agent's ability to identify objects from dynamic viewpoints in VLN scenarios by effectively integrating pre-trained VLM knowledge into the perception process, without requiring VLM fine-tuning. Our method enhances the agent's ability to interpret and respond to environmental cues while ensuring computational efficiency. Experimental results have shown that our method outperforms the baseline methods on multiple benchmarks, which validate the effectiveness, robustness and generalizability of our method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
