VL-KnG：从单目视频构建持久时空知识图谱以实现具身场景理解
Spatiotemporal Knowledge Graphs as Persistent Scene Memory for Embodied Question Answering
AI总结：
VL-KnG通过构建时空知识图谱实现高效具身场景理解，采用LLM驱动的时空对象关联和图增强检索，无需3D重建，支持常数时间推理，优于现有前沿VLMs。
摘要：
视觉-语言模型（VLMs）在图像级场景理解方面表现优异，但在处理长视频序列时往往缺乏持久记忆、显式空间表示和计算效率。我们提出VL-KnG，一种无需训练的框架，通过单目视频构建时空知识图谱，无需3D重建即可连接细粒度场景图和全局拓扑图。VL-KnG将视频分块处理，通过LLM驱动的时空对象关联（STOA）保持持久对象身份，并通过图增强检索（GER）回答查询，GER是图RAG子图检索与SigLIP2视觉定位的混合方法。一旦构建完成，知识图谱在查询时无需重新处理视频，支持无论视频长度如何的常数时间推理。在三个基准测试（OpenEQA、NaVQA和我们新引入的WalkieKnowledge）上的评估表明，VL-KnG在显著更低的查询延迟下，匹配或超越了前沿VLMs在具身场景理解任务上的性能。实际机器人部署验证了其在常数时间扩展方面的实用性。
英文摘要：
Vision-language models (VLMs) demonstrate strong image-level scene understanding, but reasoning over long egocentric video remains costly: because VLMs maintain no persistent memory or explicit spatial representation, all sampled frames must be re-processed for every new query. We present VL-KnG, a training-free framework that constructs spatiotemporal knowledge graphs from monocular egocentric video, bridging fine-grained scene graphs and global topological graphs without 3D reconstruction. VL-KnG processes video in chunks, maintains persistent object identities via large language model (LLM)-based Spatiotemporal Object Association (STOA), and answers questions through Graph-Enhanced Retrieval (GER), which combines subgraph retrieval with visual grounding. Once constructed, the knowledge graph removes the need to revisit raw video at query time, decoupling query latency from video length. Across three embodied question answering benchmarks -- OpenEQA, NaVQA, and our newly introduced WalkieKnowledge -- VL-KnG achieves accuracy competitive with frontier VLMs while answering queries at substantially lower latency and with explainable, graph-grounded reasoning; it further surpasses prior persistent-representation baselines and open-weight VLMs in several settings. Deployment on a real robot demonstrates practical applicability, with query latency remaining stable as observation history grows. As a persistent, queryable scene memory, VL-KnG provides a concrete substrate for memory maintenance and knowledge updating in embodied AI agents.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
