Tri-Bench：在相机倾斜和物体干扰下测试VLM在空间推理中的可靠性
Tri-Bench: Stress-Testing VLM Reliability on Spatial Reasoning under Camera Tilt and Object Interference
AI总结：
Tri-Bench通过测试VLM在相机倾斜和物体干扰下的空间推理可靠性，揭示了模型在几何推理中的不足。
摘要：
可验证的几何推理是可信和可控的代理AI的关键组成部分。尽管具有出色的性能，但视觉语言模型（VLMs）在现实场景变化下往往表现不佳。我们提出了Tri-Bench，一个紧凑的平面三角形问题基准，该基准隔离了相对几何推理，同时压力测试两个关键部署因素：相机姿态（平面与倾斜）和场景上下文通过物体干扰（10个日常物体）。为了测试可验证性和控制性，我们使用一个固定的提示来评估四个最近的VLMs，该提示明确描述了一个周围的正方形边框，从而通过仿射变换得到正确答案。我们评估了六个简单的任务，针对二元和连续目标，观察到相对于3D地面真实值的总体准确率较低，平均约为69%（最佳约75%，最差约64%）。同样的响应在图像平面的2D投影中更加一致，平均准确率约为72%。所有四个VLMs在识别少数形状类别（等边、等腰、直角三角形）时均失败，准确率降至约0%。此外，总体VLM准确率在相机倾斜下下降约4.1%。这表明模型无法正确利用提示中提供的显式参考框架提示，而默认使用2D图像平面线索。最后，我们发现物体干扰对VLM准确率没有显著影响。
英文摘要：
Verifiable geometric reasoning is a critical component for trustworthy and controllable agentic AI. Despite impressive capabilities, Vision-Language Models (VLMs) often fail under realistic scene changes. We present Tri-Bench, a compact benchmark of planar triangle problems that isolates relative geometric reasoning while stressing two deployment-critical factors: camera pose (planar vs. tilted) and scene context via object interference (10 everyday objects). To test verifiability and control, we evaluate four recent VLMs using a single, fixed prompt whose guardrail explicitly describes a surrounding square border, enabling correct answers via homography. We evaluate six simple tasks over binary and continuous targets, and observe that the overall accuracy with respect to 3D ground truth is modest, ~69% on average (best ~75%, worst ~64%). The same responses align even more closely with 2D projections in the image plane, where mean accuracy is ~72%. All four VLMs consistently fail, with accuracy falling to ~0%, on recognizing minority shape classes (equilateral, isosceles, right-angled triangles). Additionally, overall VLM accuracy degrades by ~4.1% under camera tilt. This demonstrates that models fail to correctly utilize the explicit frame-of-reference hint provided in the prompt and default to 2D image plane cues. Finally, we find that object interference has no significant effect on VLM accuracy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
