CrashChat: 一种多模态大语言模型用于多任务交通事故视频分析
CrashChat: A Multimodal Large Language Model for Multitask Traffic Crash Video Analysis
AI总结：
CrashChat是一种多模态大语言模型，用于多任务交通事故视频分析，通过任务解耦和分组策略提升事故识别、定位等任务的性能。
摘要：
自动化交通事故视频分析对于利用日益增长的驾驶视频数据用于交通安全管理及自动驾驶责任归属至关重要。交通事故视频分析是一个具有挑战性的多任务问题，由于视频数据中事故事件的复杂时空动态和多样的分析需求。它需要涵盖事故识别、时间定位和高级视频理解的能力。然而，现有模型无法在一个统一的框架内完成所有这些任务，且有效的训练策略仍待探索。为此，本文提出CrashChat，一种基于VideoLLaMA3的多任务交通事故分析多模态大语言模型（MLLM）。CrashChat通过指令微调获取领域特定知识，并采用基于任务解耦和分组的新型多任务学习策略，最大化任务组内和跨任务组的联合学习效益，同时缓解负迁移。在整合的公共数据集上的数值实验表明，CrashChat在模型规模和传统基于视觉的方法上均优于现有MLLMs，实现了最先进的性能。它在事故识别中达到近完美的准确率，事故定位的提升达176%，在更具挑战性的预事故定位中提升40%。与通用MLLMs相比，它在事故描述和推理任务中显著提高了文本准确性和内容覆盖度，BLEU分数增加0.18-0.41，ROUGE分数增加0.18-0.42。除了强大的性能，CrashChat还是一种方便的端到端分析工具，可用于实际应用。CrashChat的数据集和实现代码可在https://github.com/Liangkd/CrashChat上获得。
英文摘要：
Automating crash video analysis is essential to leverage the growing availability of driving video data for traffic safety research and accountability attribution in autonomous driving. Crash video analysis is a challenging multitask problem due to the complex spatiotemporal dynamics of crash events in video data and the diverse analytical requirements involved. It requires capabilities spanning crash recognition, temporal grounding, and high-level video understanding. Existing models, however, cannot perform all these tasks within a unified framework, and effective training strategies for such models remain underexplored. To fill these gaps, this paper proposes CrashChat, a multimodal large language model (MLLM) for multitask traffic crash analysis, built upon VideoLLaMA3. CrashChat acquires domain-specific knowledge through instruction fine-tuning and employs a novel multitask learning strategy based on task decoupling and grouping, which maximizes the benefit of joint learning within and across task groups while mitigating negative transfer. Numerical experiments on consolidated public datasets demonstrate that CrashChat consistently outperforms existing MLLMs across model scales and traditional vision-based methods, achieving state-of-the-art performance. It reaches near-perfect accuracy in crash recognition, a 176\% improvement in crash localization, and a 40\% improvement in the more challenging pre-crash localization. Compared to general MLLMs, it substantially enhances textual accuracy and content coverage in crash description and reasoning tasks, with 0.18-0.41 increases in BLEU scores and 0.18-0.42 increases in ROUGE scores. Beyond its strong performance, CrashChat is a convenient, end-to-end analytical tool ready for practical implementation. The dataset and implementation code for CrashChat are available at https://github.com/Liangkd/CrashChat.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
