面向安全驾驶感知的溯因自我视角事故视频理解
Abductive Ego-View Accident Video Understanding for Safe Driving Perception
AI总结:
该研究构建含11,727个自我视角事故视频的MM-AU数据集,并提出AdVersa-SD框架,利用溯因CLIP与OAVD挖掘事故因果链,提升安全驾驶感知。
摘要:
我们提出了 MM-AU,这是一个用于多模态事故视频理解(Multi-Modal Accident video Understanding)的新型数据集。MM-AU 包含 11,727 个真实场景中的自我视角事故视频,每个视频都配有时间对齐的文本描述。我们标注了超过 223 万个对象框以及 58,650 对基于视频的事故原因,涵盖 58 个事故类别。MM-AU 支持多种事故理解任务,尤其是通过多模态视频扩散来理解安全驾驶中的事故因果链。基于 MM-AU,我们提出了用于安全驾驶感知的溯因式事故视频理解框架 AdVersa-SD。AdVersa-SD 通过以对象为中心的视频扩散(Object-Centric Video Diffusion,OAVD)方法执行视频扩散,该方法由溯因 CLIP 模型驱动。该模型引入对比交互损失,用于学习正常、接近事故和事故帧与相应文本描述(如事故原因、预防建议和事故类别)之间的成对共现关系。OAVD 在视频生成中固定原始帧背景内容,同时强化因果区域学习,以找到特定事故的主导因果链。大量实验验证了 AdVersa-SD 的溯因能力,以及 OAVD 相对于最先进扩散模型的优越性。此外,由于 AdVersa-SD 依赖精确的对象和事故原因信息,我们还针对目标检测和事故原因问答提供了审慎的基准评估。
英文摘要:
We present MM-AU, a novel dataset for Multi-Modal Accident video Understanding. MM-AU contains 11,727 in-the-wild ego-view accident videos, each with temporally aligned text descriptions. We annotate over 2.23 million object boxes and 58,650 pairs of video-based accident reasons, covering 58 accident categories. MM-AU supports various accident understanding tasks, particularly multimodal video diffusion to understand accident cause-effect chains for safe driving. With MM-AU, we present an Abductive accident Video understanding framework for Safe Driving perception (AdVersa-SD). AdVersa-SD performs video diffusion via an Object-Centric Video Diffusion (OAVD) method which is driven by an abductive CLIP model. This model involves a contrastive interaction loss to learn the pair co-occurrence of normal, near-accident, accident frames with the corresponding text descriptions, such as accident reasons, prevention advice, and accident categories. OAVD enforces the causal region learning while fixing the content of the original frame background in video generation, to find the dominant cause-effect chain for certain accidents. Extensive experiments verify the abductive ability of AdVersa-SD and the superiority of OAVD against the state-of-the-art diffusion models. Additionally, we provide careful benchmark evaluations for object detection and accident reason answering since AdVersa-SD relies on precise object and accident reason information.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。