用于驾驶员行为分析中长形式推理的多帧视觉-语言模型
Multi-Frame Vision-Language Model for Long-form Reasoning in Driver Behavior Analysis
AI总结：
本文提出一个多模态指令调优数据集与驾驶员辅导推理系统，利用多帧视觉-语言模型对行车记录仪视频进行长形式推理，以识别危险驾驶行为并支持商用驾驶员辅导。
摘要：
在现实场景中识别危险驾驶行为对驾驶员和行人的安全至关重要。然而，将自然语言模型整合到该领域的工作仍相对未被开发。为解决这一问题，我们创建了一个新颖的多模态指令调优数据集和驾驶员辅导推理系统。我们的主要用例是基于行车记录仪的商用驾驶员辅导。北美行车记录仪市场预计在2022年至2027年间将实现15.4%的复合年增长率。我们的数据集使语言模型能够学习各种危险驾驶场景下的视觉指令，强调对有效驾驶员辅导和管理层理解至关重要的详细推理。我们的模型在面向道路和面向驾驶员的RGB摄像头画面上进行训练，捕捉配备行车记录仪的车辆中驾驶行为的全面范围。
英文摘要：
Identifying risky driving behavior in real-world situations is essential for the safety of both drivers and pedestrians. However, integrating natural language models in this field remains relatively untapped. To address this, we created a novel multi-modal instruction tuning dataset and driver coaching inference system. Our primary use case is dashcam-based coaching for commercial drivers. The North American Dashcam Market is expected to register a CAGR of 15.4 percent from 2022 to 2027. Our dataset enables language models to learn visual instructions across various risky driving scenarios, emphasizing detailed reasoning crucial for effective driver coaching and managerial comprehension. Our model is trained on road-facing and driver-facing RGB camera footage, capturing the comprehensive scope of driving behavior in vehicles equipped with dashcams.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
