arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.15237cs.HC
收藏

视觉语言模型中的真实深度:用于虚拟现实机器人支持的紧急急救中的空间情境理解

Ground-Truth Depth in Vision Language Models: Spatial Context Understanding in Conversational AI for XR-Robotic Support in Emergency First Response

AIT - Austrian Institute of Technology(奥地利技术研究所)

Rodrigo Gutierrez Maquilon, Marita Hueber, Georg Regal, Manfred Tscheligi

AI总结:

本文提出了一种结合深度传感与视觉语言模型的原型,用于增强紧急急救中的空间情境理解,通过深度增强提高准确性和稳定性,同时降低工作量。

摘要:

大型语言模型(LLMs)正在越来越多地用于紧急急救(EFR)应用中,以支持态势感知(SA)和决策制定,但大多数模型仅基于文本或2D图像,对核心EFR SA能力如空间推理支持有限。我们通过评估一个原型来填补这一空白,该原型融合了机器人-mounted深度传感和YOLO检测,与能够口头描述检测对象度量基础距离的视觉语言模型(VLM)相结合(例如,椅子距离3.02米)。在混合现实毒烟场景中,参与者在三种条件下估计受害者和出口窗户的距离:视频-only、深度无感知VLM和深度增强VLM。深度增强提高了客观准确性和稳定性,例如受害者和窗户距离估计误差降低,同时提高了态势感知而不增加工作量。相反,深度无感知的辅助增加了工作量并略微降低了准确性。我们通过证明度量基础、以对象为中心的口头信息支持EFR中的空间推理,并在时间压力下改进决策相关判断,为人类SA增强做出了贡献。

英文摘要:

Large language models (LLMs) are increasingly used in emergency first response (EFR) applications to support situational awareness (SA) and decision-making, yet most operate on text or 2D imagery and offer little support for core EFR SA competencies like spatial reasoning. We address this gap by evaluating a prototype that fuses robot-mounted depth sensing and YOLO detection with a vision language model (VLM) capable of verbalizing metrically-grounded distances of detected objects (e.g., the chair is 3.02 meters away). In a mixed-reality toxic-smoke scenario, participants estimated distances to a victim and an exit window under three conditions: video-only, depth-agnostic VLM, and depth-augmented VLM. Depth-augmentation improved objective accuracy and stability, e.g., the victim and window distance estimation error dropped, while raising situational awareness without increasing workload. Conversely, depth- agnostic assistance increased workload and slightly worsened accuracy. We contribute to human SA augmentation by demonstrating that metrically grounded, object-centric verbal information supports spatial reasoning in EFR and improves decision-relevant judgments under time pressure.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑