通过用密集CLIP泛化语义映射实现开放词汇目标导航
Open-Vocabulary Object-Goal Navigation by Generalizing Semantic Mapping with Dense CLIP
AI总结：
研究面向对象的实体导航任务，提出OVExp框架，利用密集CLIP模型展示基于语义地图的目标预测网络泛化能力，设计跨模态转移策略解决训练成本高问题，在ObjectNav基准上对未知目标有强大泛化能力。
摘要：
面向对象的实体导航任务要求智能体在未知环境中定位特定对象，这些对象可以通过类别或图像定义。虽然近期方法在利用基础模型将封闭集模型扩展到开放词汇场景方面取得了进展，但它们通常依赖无需训练的大语言模型或端到端强化学习微调。然而，它们在效率（如大语言模型推理的开销和成本）以及强化学习密集训练的有限泛化能力方面面临挑战。本文提出了OVExp，一个用于开放词汇探索的高效训练框架。我们首次展示了使用密集CLIP模型的基于语义地图的目标预测网络的泛化能力。一个主要挑战是在语义地图中保留精确的逐点对象位置和可泛化的视觉表示会导致训练成本过高。为解决此问题，我们设计了一种语义映射上的跨模态转移策略，该策略采用有趣的仅文本训练并在测试时转移到多模型语义映射和目标。尽管依赖具有有限对象的基于文本的空间布局，但OVExp在既定的ObjectNav基准上展示了对未知目标的强大泛化能力。
英文摘要：
Object-oriented embodied navigation tasks require agents to locate specific objects, either defined by category or images, in unseen environments. While recent methods have made progress in extending closed-set models to open-vocabulary scenarios with foundation models, they typically rely on training-free large language models (LLMs) or finetuning with end-to-end reinforcement learning (RL). However, they face challenges in efficiency (e.g., the overhead and cost of LLM inference) and limited generalization from intensive RL training. In this paper, we propose OVExp, a training-efficient framework for open-vocabulary exploration. We make the first effort to demonstrate the generalization capabilities of semantic map-based goal prediction networks using Dense CLIP models. A major challenge is that preserving both precise point-wise object locations and generalizable visual representations in the semantic map leads to unaffordable training costs. To address this, we design a Cross-Modal Transfer on Semantic Mapping strategy which adapts an intriguing text-only training and transfer to multi-model semantic mapping and goals in test-time. Despite relying on text-based spatial layouts with limited objects, OVExp demonstrates robust generalization to unseentargets on established ObjectNav benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
