FindingDory：一个评估具身智能体记忆能力的基准
FindingDory: A Benchmark to Evaluate Memory in Embodied Agents
AI总结：
针对VLM难以利用跨多日海量图像经验的问题，本文提出Habitat中的FindingDory基准，通过60项可扩展长程具身任务评估记忆、推理与行动整合能力，并给出VLM结合导航策略的基线。
摘要：
大型视觉-语言模型近来在规划与控制任务中展现出令人瞩目的性能，推动了人们将其应用于真实世界机器人技术的兴趣。然而，将这些模型部署到具身情境中进行推理，受到其整合跨越多日收集、并由海量图像集合表征的长期经验能力的限制。当前VLM通常难以同时处理超过数百张图像，这凸显出在具身环境中需要更高效的机制来处理长期记忆。为有效评估这些模型用于长时程控制的能力，基准必须专门针对记忆对成功至关重要的场景。现有长视频问答基准忽视了物体操控和导航等具身挑战，而这些挑战需要低级技能以及对过往交互的细粒度推理。此外，具身智能体中有效的记忆整合既包括回忆相关历史信息，也包括基于该信息执行动作，因此必须将这些方面结合起来研究，而不是孤立研究。在本工作中，我们在Habitat模拟器中引入了一个面向长程具身任务的新基准。该基准在60项任务中评估基于记忆的能力，这些任务要求在环境中保持持续参与和情境感知。这些任务还可以通过程序化方式扩展为更长、更具挑战性的版本，从而实现对记忆和推理的可扩展评估。我们还提出了将最先进VLM与低级导航策略相集成的基线，评估它们在这些记忆密集型任务上的表现，并指出有待改进的领域。
英文摘要：
Large vision-language models have recently demonstrated impressive performance in planning and control tasks, driving interest in their application to real-world robotics. However, deploying these models for reasoning in embodied contexts is limited by their ability to incorporate long-term experience collected across multiple days and represented by vast collections of images. Current VLMs typically struggle to process more than a few hundred images concurrently, highlighting the need for more efficient mechanisms to handle long-term memory in embodied settings. To effectively evaluate these models for long-horizon control, a benchmark must specifically target scenarios where memory is crucial for success. Existing long-video QA benchmarks overlook embodied challenges like object manipulation and navigation, which demand low-level skills and fine-grained reasoning over past interactions. Moreover, effective memory integration in embodied agents involves both recalling relevant historical information and executing actions based on that information, making it essential to study these aspects together rather than in isolation. In this work, we introduce a new benchmark for long-range embodied tasks in the Habitat simulator. This benchmark evaluates memory-based capabilities across 60 tasks requiring sustained engagement and contextual awareness in an environment. The tasks can also be procedurally extended to longer and more challenging versions, enabling scalable evaluation of memory and reasoning. We also present baselines that integrate state-of-the-art VLMs with low level navigation policies, assessing their performance on these memory-intensive tasks and highlight areas for improvement.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
