FlySearch：探索视觉-语言模型如何进行探索
FlySearch: Exploring how vision-language models explore
AI总结：
本文提出面向户外复杂场景目标搜索与导航的照片级真实 3D 基准 FlySearch，系统评估先进 VLM 的主动探索能力，揭示其与人类表现的差距及主要失败原因。
摘要：
现实世界杂乱且非结构化。揭示关键信息通常需要主动的、目标驱动的探索。视觉-语言模型（VLM）近来已成为许多困难任务中流行的零样本工具，但它们能否在这种条件下有效运作，仍有待观察。本文通过提出 FlySearch 来回答这一问题，它是一个用于在复杂场景中搜索目标物体并导航至其位置的三维、户外、照片级真实环境。作者定义了三组难度不同的场景，并观察到最先进的 VLM 即使在最简单的探索任务上也无法可靠完成，而且随着任务难度增加，它们与人类表现之间的差距也在扩大。论文识别出一系列核心原因，涵盖视觉幻觉、上下文误解以及任务规划失败，并表明其中部分问题可以通过微调解决。作者公开发布了该基准、场景及底层代码库。
英文摘要：
The real world is messy and unstructured. Uncovering critical information often requires active, goal-driven exploration. It remains to be seen whether Vision-Language Models (VLMs), which recently emerged as a popular zero-shot tool in many difficult tasks, can operate effectively in such conditions. In this paper, we answer this question by introducing FlySearch, a 3D, outdoor, photorealistic environment for searching and navigating to objects in complex scenes. We define three sets of scenarios with varying difficulty and observe that state-of-the-art VLMs cannot reliably solve even the simplest exploration tasks, with the gap to human performance increasing as the tasks get harder. We identify a set of central causes, ranging from vision hallucination, through context misunderstanding, to task planning failures, and we show that some of them can be addressed by finetuning. We publicly release the benchmark, scenarios, and the underlying codebase.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
