探索视觉-语言模型中的碰撞接地以实现安全的人机协作
Probing Collision Grounding in Vision-Language Models for Safe Human-Robot Collaboration
AI总结：
针对安全人机协作，提出碰撞接地概念及物理基准TouchSafeBench，评估视觉-语言模型在分类当前安全状态和预警即将碰撞任务中的表现，发现现有模型不可靠，视觉流畅性不等于物理责任性。
摘要：
安全的人机协作需要的不仅仅是视觉描述：监控器必须确定机器人身体是否安全分离、已经与场景或人发生碰撞，或即将碰撞。我们将这种能力称为碰撞接地：将视觉观察与机器人身体几何、相机视角、场景布局、人体接近度和时间运动相结合，以推断当前和即将发生的接触。我们引入了TouchSafeBench，一个基于物理的基准，用于评估视觉-语言模型（VLM）中的碰撞接地能力。TouchSafeBench基于Habitat 3.0构建，包含2,940个模拟室内共现场景，涵盖社交导航和社交重排，具有同步的多视角RGB-D观测、自上而下的轨迹地图、校准的相机元数据和模拟器导出的接触标签。我们研究了两个面向部署的任务：分类当前安全状态和在接触前预警即将发生的碰撞。在三个前沿或面向机器人的VLM和九种视觉表示中，当前模型远未达到可靠：最佳平均Macro-F1仍低于50%，显式深度不会自动转化为机器人身体碰撞证据，且机器人与场景的接触始终比人与人的接触风险更难。TouchSafeBench揭示了具身VLM的一个核心限制：视觉流畅性并不意味着物理责任性。可靠的机器人安全监控器需要能够显式绑定视角、机器人形态、度量几何和未来碰撞的表示。我们将在论文被接收后发布该基准。
英文摘要：
Safe human--robot collaboration requires more than visual description: a monitor must determine whether the robot body is safely separated, already colliding with the scene or a person, or about to collide. We call this capability collision grounding: binding visual observations to robot body geometry, camera viewpoint, scene layout, human proximity, and temporal motion in order to infer present and imminent contact. We introduce TouchSafeBench, a physics-grounded benchmark for evaluating collision grounding in vision-language models (VLMs). Built in Habitat~3.0, TouchSafeBench contains 2,940 simulated indoor co-presence episodes across social navigation and social rearrangement, with synchronized multi-view RGB-D observations, top-down trajectory maps, calibrated camera metadata, and simulator-derived contact labels. We study two deployment-facing tasks: classifying the current safety state and warning about imminent collision before contact. Across three frontier or robotics-oriented VLMs and nine visual representations, current models remain far from reliable: the best average Macro-F1 stays below 50\%, explicit depth is not automatically transformed into robot-body collision evidence, and robot--scene contact is consistently harder than human-contact risk. TouchSafeBench reveals a central limitation of embodied VLMs: visual fluency does not imply physical accountability. Reliable robot safety monitors will need representations that explicitly bind viewpoint, robot morphology, metric geometry, and future collision. We will release the benchmark upon acceptance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
