基于视频的野外车辆监控：利用自反思视觉语言模型进行车牌、品牌和型号识别
Video-based Vehicle Surveillance in the Wild: License Plate, Make, and Model Recognition with Self Reflective Vision-Language Models
AI总结：
本研究利用自反思视觉语言模型处理手持或车载视频中的车牌及车辆品牌型号识别，在自建数据集上分别取得91.67%和66.67%的top-1准确率，自反思模块平均提升5.72%。
摘要：
自动车牌识别（ALPR）和车辆品牌型号识别是智能交通系统的基石，支持执法、收费和事后调查等应用。与固定安装相比，将这些方法应用于手持智能手机或非静态车载摄像头拍摄的视频，面临着独特的挑战，包括频繁的相机运动、多变的视角、遮挡以及未知的道路几何形状。传统的ALPR解决方案依赖于专用硬件和手工设计的OCR流水线，在这些条件下往往会性能下降。近期大型视觉语言模型（VLM）的进展使得能够直接从任意图像中识别文本和语义属性。本研究评估了VLM在利用手持智能手机和非静态车载摄像头拍摄的单目视频中进行ALPR以及品牌和型号识别的潜力。所提出的车牌识别流水线先筛选出清晰帧，然后使用多种提示策略向VLM发送多模态提示。品牌和型号识别流水线使用相同的VLM，但采用修改后的提示，并可选配自反思模块。在自反思模块中，模型将查询图像与来自134类数据集中的参考图像进行对比，纠正不匹配之处。在德克萨斯大学奥斯汀分校校园内收集的智能手机数据集上的实验，ALPR的top-1准确率达到91.67%，品牌和型号识别达到66.67%。在公开的UFPR-ALPR数据集上，该方法分别达到了83.05%和61.07%的准确率。自反思模块进一步将品牌和型号识别的结果平均提高了5.72%。这些发现表明，VLM为可扩展的、运动中的交通视频分析提供了一种经济高效的解决方案。
英文摘要：
Automatic license plate recognition (ALPR) and vehicle make and model recognition underpin intelligent transportation systems, supporting law enforcement, toll collection, and post-incident investigation. Applying these methods to videos captured by handheld smartphones or non-static vehicle-mounted cameras presents unique challenges compared to fixed installations, including frequent camera motion, varying viewpoints, occlusions, and unknown road geometry. Traditional ALPR solutions, dependent on specialized hardware and handcrafted OCR pipelines, often degrade under these conditions. Recent advances in large vision-language models (VLMs) enable direct recognition of textual and semantic attributes from arbitrary imagery. This study evaluates the potential of VLMs for ALPR and makes and models recognition using monocular videos captured with handheld smartphones and non-static mounted cameras. The proposed license plate recognition pipeline filters to sharp frames, then sends a multimodal prompt to a VLM using several prompt strategies. Make and model recognition pipeline runs the same VLM with a revised prompt and an optional self-reflection module. In the self-reflection module, the model contrasts the query image with a reference from a 134-class dataset, correcting mismatches. Experiments on a smartphone dataset collected on the campus of the University of Texas at Austin, achieve top-1 accuracies of 91.67% for ALPR and 66.67% for make and model recognition. On the public UFPR-ALPR dataset, the approach attains 83.05% and 61.07%, respectively. The self-reflection module further improves results by 5.72% on average for make and model recognition. These findings demonstrate that VLMs provide a cost-effective solution for scalable, in-motion traffic video analysis.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
