面向端到端事件分析的并行密集视频字幕提升交通安全性能
Enhancing Traffic Safety with Parallel Dense Video Captioning for End-to-End Event Analysis
AI总结：
本研究针对2024年AI City挑战赛交通安全分析任务，基于PDVC框架结合CLIP特征、领域适配与知识迁移优化密集视频字幕，在WTS数据集测试集上获竞赛第6名。
摘要：
本文介绍了我们在2024年AI City挑战赛第二赛道的解决方案。该任务旨在利用Woven Traffic Safety（WTS，一个用于细粒度时空理解的真实世界行人中心交通视频数据集）解决交通安全描述与分析问题。我们的方案主要围绕以下几点展开：1）针对密集视频字幕任务，我们采用带并行解码的密集视频字幕框架（PDVC）对视觉-语言序列建模，按章节生成视频的密集字幕。2）利用CLIP提取视觉特征，更高效地实现视觉与文本表征间的跨模态训练。3）开展领域特定的模型适配，缓解给视频理解带来识别挑战的域偏移问题。4）此外，借助BDD-5K带字幕视频进行知识迁移，以更好地理解WTS视频并生成更准确的字幕。我们的方案在测试集上取得了竞赛第6名的成绩，开源代码将发布于https://github.com/UCF-SST-Lab/AICity2024CVPRW。
英文摘要：
This paper introduces our solution for Track 2 in AI City Challenge 2024. The task aims to solve traffic safety description and analysis with the dataset of Woven Traffic Safety (WTS), a real-world Pedestrian-Centric Traffic Video Dataset for Fine-grained Spatial-Temporal Understanding. Our solution mainly focuses on the following points: 1) To solve dense video captioning, we leverage the framework of dense video captioning with parallel decoding (PDVC) to model visual-language sequences and generate dense caption by chapters for video. 2) Our work leverages CLIP to extract visual features to more efficiently perform cross-modality training between visual and textual representations. 3) We conduct domain-specific model adaptation to mitigate domain shift problem that poses recognition challenge in video understanding. 4) Moreover, we leverage BDD-5K captioned videos to conduct knowledge transfer for better understanding WTS videos and more accurate captioning. Our solution has yielded on the test set, achieving 6th place in the competition. The open source code will be available at https://github.com/UCF-SST-Lab/AICity2024CVPRW
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
