通过视觉语言模型从自我中心视觉解码行人过街意图
Decoding Pedestrian Crossing Intention from Egocentric Vision via Vision Language Models
AI总结：
利用视觉语言模型（VLM）将行人过街意图预测转化为视觉问答任务，通过参数高效微调并结合自我运动、车辆运动和眼动等上下文线索，在自我中心视频上实现了14.5%的准确率提升，创下新纪录。
摘要：
自我中心视觉提供了人类感知和决策的第一人称视角，但其在交通安全预测方面的潜力尚未得到充分探索。在这项工作中，我们研究从短自我中心视频片段中解码行人过街意图。我们通过将任务表述为封闭式视觉问答（VQA）问题，并利用视觉语言模型（VLM）来预测行人的意图。我们首先在零样本设置下对三个系列的最先进VLM进行了基准测试，发现它们相对于随机猜测有适度提升，但表现出有限的高层次交通推理能力。基于这些发现，我们进一步使用参数高效微调将VLM适应于目标任务。我们的结果表明，微调后的模型显著优于其零样本对应模型，并在专门的基于Transformer的基线基础上实现了9%的准确率提升。最后，我们证明加入额外的上下文线索，包括自我运动、车辆运动和眼动，进一步提高了预测性能。特别是，由眼动和自我运动引导的微调Qwen3-VL-2B模型相比Transformer基线实现了14.5%的准确率提升，为自我中心行人意图解码建立了新的最先进水平。
英文摘要：
Egocentric vision offers a first-person view of human perception and decision making, yet its potential for traffic-safety prediction remains underexplored. In this work, we study the decoding of pedestrian crossing intentions from short egocentric video clips. We approach this by formulating the task as a closed-ended visual question answering (VQA) problem and leveraging vision language models (VLMs) to predict the pedestrians' intent. We first benchmark three families of state-of-the-art VLMs in a zero-shot setting, finding that they achieve moderate gains over random guessing but exhibit limited higher-level traffic reasoning. Motivated by these findings, we further adapt VLMs to the target task using parameter-efficient fine-tuning. Our results show that the fine-tuned models substantially outperform their zero-shot counterparts and achieve a 9\% accuracy improvement over a specialized transformer-based baseline. Finally, we demonstrate that incorporating additional contextual cues, including ego motion, vehicle motion, and eye gaze, further improves predictive performance. In particular, the fine-tuned Qwen3-VL-2B model guided by eye gaze and ego motion achieves a 14.5% accuracy improvement over the transformer baseline, establishing a new state of the art for egocentric pedestrian intent decoding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
