R4：在4D时空空间中为视觉语言模型引入检索增强推理
R4: Retrieval-Augmented Reasoning for Vision-Language Models in 4D Spatio-Temporal Space
AI总结：
R4通过构建4D时空知识库，使视觉语言模型具备结构化终身记忆，实现无需训练的检索增强推理，提升动态环境中的具身推理能力。
摘要：
人类通过构建持久且结构化的内部表示来感知和推理周围环境，这些表示编码了语义意义、空间布局和时间动态。这些多模态记忆使他们能够回忆过去事件、推断未观察到的状态，并将新信息整合到情境依赖的推理中。受这一能力的启发，我们引入了R4，一种无需训练的框架，用于在4D时空空间中实现检索增强推理，使视觉语言模型（VLMs）具备结构化、终身记忆。R4通过在度量空间和时间中锚定对象级语义描述，持续构建一个4D知识库，从而生成一个可以跨代理共享的持久世界模型。在推理过程中，自然语言查询被分解为语义、空间和时间键，以检索相关观察，这些观察被整合到VLM的推理中。与经典检索增强生成方法不同，R4在4D空间中直接进行检索，使事件性和协作推理成为可能，而无需训练。在具身问答和导航基准测试中，R4在时空信息检索和推理方面显著优于基线，推动了在动态环境中进行具身4D推理的新范式。
英文摘要：
Humans perceive and reason about their surroundings in four dimensions by building persistent, structured internal representations that encode semantic meaning, spatial layout, and temporal dynamics. These multimodal memories enable them to recall past events, infer unobserved states, and integrate new information into context-dependent reasoning. Inspired by this capability, we introduce R4, a training-free framework for retrieval-augmented reasoning in 4D spatio-temporal space that equips vision-language models (VLMs) with structured, lifelong memory. R4 continuously constructs a 4D knowledge database by anchoring object-level semantic descriptions in metric space and time, yielding a persistent world model that can be shared across agents. At inference, natural language queries are decomposed into semantic, spatial, and temporal keys to retrieve relevant observations, which are integrated into the VLM's reasoning. Unlike classical retrieval-augmented generation methods, retrieval in R4 operates directly in 4D space, enabling episodic and collaborative reasoning without training. Experiments on embodied question answering and navigation benchmarks demonstrate that R4 substantially improves retrieval and reasoning over spatio-temporal information compared to baselines, advancing a new paradigm for embodied 4D reasoning in dynamic environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
