arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2502.20573cs.CVcs.CL
收藏

城市交叉口的视觉推理:微调GPT-4o用于交通冲突检测

Visual Reasoning at Urban Intersections: FineTuning GPT-4o for Traffic Conflict Detection

Arab American University(阿拉伯美国大学);Queensland University of Technology(昆士兰科技大学);CARRS-Q;Centre for Data Science(数据科学中心)

Sari Masri, Huthaifa I. Ashqar, Mohammed Elhenawy

AI总结:

本研究微调GPT-4o利用鸟瞰视频检测无信号灯交叉口的交通冲突,准确率达77.14%,解释和行动建议准确率分别达89.9%和92.3%,证明MLLMs用于实时交通管理的可行性。

摘要:

无信号灯的城市交叉口交通控制由于复杂性、频繁冲突和盲区而面临重大挑战。本研究探索利用多模态大语言模型(MLLMs)(如GPT-4o)的能力,通过直接使用四岔交叉口的鸟瞰视频,提供逻辑和视觉推理。在该方法中,GPT-4o作为智能系统检测冲突,并为驾驶员提供解释和建议。微调后的模型达到了77.14%的准确率,而对微调后的GPT-4o真实预测值的人工评估显示,模型生成的解释准确率为89.9%,推荐的下一步行动准确率为92.3%。这些结果凸显了使用MLLMs以视频作为输入进行实时交通管理的可行性,为交叉口交通管理和运营提供了可扩展且可操作的见解。本研究中使用的代码可在https://github.com/sarimasri3/Traffic-Intersection-Conflict-Detection-using-images.git获取。

英文摘要:

Traffic control in unsignalized urban intersections presents significant challenges due to the complexity, frequent conflicts, and blind spots. This study explores the capability of leveraging Multimodal Large Language Models (MLLMs), such as GPT-4o, to provide logical and visual reasoning by directly using birds-eye-view videos of four-legged intersections. In this proposed method, GPT-4o acts as intelligent system to detect conflicts and provide explanations and recommendations for the drivers. The fine-tuned model achieved an accuracy of 77.14%, while the manual evaluation of the true predicted values of the fine-tuned GPT-4o showed significant achievements of 89.9% accuracy for model-generated explanations and 92.3% for the recommended next actions. These results highlight the feasibility of using MLLMs for real-time traffic management using videos as inputs, offering scalable and actionable insights into intersections traffic management and operation. Code used in this study is available at https://github.com/sarimasri3/Traffic-Intersection-Conflict-Detection-using-images.git.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑