看见车辆之所见：面向物理自动驾驶车辆的视频增强虚拟现实

中文摘要 AI 辅助

自动驾驶汽车有望提高道路安全性和效率，但乘客往往不确定车辆感知到了什么以及为何如此行动。虚拟现实（VR）提供了一种安全且可重复的媒介来呈现这些信息，然而大多数面向乘客的VR研究依赖于完全模拟的车辆或预先编写的场景，因此展示给用户的运动和感知并非源自物理运行的自动驾驶系统。本文提出了一种视频增强的VR框架，该框架将物理ROS 2自主机器人车辆与部署在Meta Quest 3S头显上的Unity 6应用相耦合。车辆状态和实时车载摄像头流通过两个独立的通信信道传输，使虚拟车辆能够镜像物理机器人的运动，同时乘客通过车内仪表板界面同时查看车辆的第一人称摄像头画面及其导航决策。我们在20次重复的闭环导航试验中评估了该框架。系统实现了平均状态更新延迟29.63毫秒，物理与虚拟车辆之间的平均相对路线进度误差为2.28%，视频传输速率为每秒10.006帧，帧丢失率为0.25%。所有监测到的导航决策均在VR界面中正确反映，没有遗漏或错误通知。结果表明，该框架能够支持时间同步、语义一致且准确的路线进度表示，用于对物理自动驾驶车辆行为的沉浸式观察。

英文摘要

Autonomous vehicles are expected to improve road safety and efficiency, but passengers often remain uncertain about what the vehicle perceives and why it acts as it does. Virtual reality (VR) offers a safe and repeatable medium for presenting this information, yet most passenger-facing VR studies rely on fully simulated vehicles or pre-scripted scenarios, so the motion and perception shown to the user do not originate from a physically operating autonomous system. This paper presents a video-augmented VR framework that couples a physical ROS 2 autonomous robot vehicle to a Unity 6 application deployed on a Meta Quest 3S headset. The vehicle state and live onboard camera stream are transmitted over two independent communication channels, allowing the virtual vehicle to mirror the physical robot's motion while the passenger simultaneously views the vehicle's first-person camera feed and its navigation decisions through an in-vehicle dashboard interface. We evaluate the framework over 20 repeated closed-loop navigation trials. The system achieves a mean state-update latency of 29.63 ms, a mean relative route-progress error of 2.28% between the physical and virtual vehicles, and video delivery at 10.006 frames per second with 0.25% frame loss. All monitored navigation decisions were correctly reflected in the VR interface with no missed or incorrect notifications. The results indicate that the framework can support temporally synchronized, semantically consistent, and accurate route-progress representation for immersive observation of physical autonomous-vehicle behavior.

发表机构

• Central Michigan University（中密歇根大学）

机构由 AI 辅助整理，请以论文原文为准。