神经哨兵：用于车牌识别的统一视觉语言模型（VLM）与人类在循环持续学习
Neural Sentinel: Unified Vision Language Model (VLM) for License Plate Recognition with Human-in-the-Loop Continual Learning
AI总结：
Neural Sentinel利用视觉语言模型实现车牌识别与多任务学习，提升准确率并减少系统复杂性。
摘要：
传统自动车牌识别（ALPR）系统采用多阶段流水线，包括对象检测网络后接单独的光学字符识别（OCR）模块，导致累积误差、延迟增加和架构复杂性。本研究提出了Neural Sentinel，一种新颖的统一方法，利用视觉语言模型（VLMs）通过单次前向传递进行车牌识别、状态分类和车辆属性提取。我们的主要贡献在于证明经过微调的PaliGemma 3B模型，通过低秩适应（LoRA）进行调整，能够同时回答关于车辆图像的多个视觉问题，达到92.3%的车牌识别准确率，比EasyOCR提高了14.1%，比PaddleOCR基线提高了9.9%。我们引入了人类在循环（HITL）持续学习框架，通过用户纠正的同时防止灾难性遗忘，通过经验回放保持原始训练数据与纠正样本的70:30比例。系统实现了平均推断延迟152毫秒，预期校准误差（ECE）为0.048，表明置信度估计良好。此外，VLM首次架构能够零样本泛化到辅助任务，包括车辆颜色检测（89%）、安全带检测（82%）和乘客计数（78%），而无需特定任务训练。通过在真实世界收费站图像上的广泛实验，我们证明了统一视觉语言方法在ALPR系统中代表了一种范式转变，提供了更高的准确性、减少的架构复杂性和传统流水线方法无法实现的新兴多任务能力。
英文摘要：
Traditional Automatic License Plate Recognition (ALPR) systems employ multi-stage pipelines consisting of object detection networks followed by separate Optical Character Recognition (OCR) modules, introducing compounding errors, increased latency, and architectural complexity. This research presents Neural Sentinel, a novel unified approach that leverages Vision Language Models (VLMs) to perform license plate recognition, state classification, and vehicle attribute extraction through a single forward pass. Our primary contribution lies in demonstrating that a fine-tuned PaliGemma 3B model, adapted via Low-Rank Adaptation (LoRA), can simultaneously answer multiple visual questions about vehicle images, achieving 92.3% plate recognition accuracy, which is a 14.1% improvement over EasyOCR and 9.9% improvement over PaddleOCR baselines. We introduce a Human-in-the-Loop (HITL) continual learning framework that incorporates user corrections while preventing catastrophic forgetting through experience replay, maintaining a 70:30 ratio of original training data to correction samples. The system achieves a mean inference latency of 152ms with an Expected Calibration Error (ECE) of 0.048, indicating well calibrated confidence estimates. Additionally, the VLM first architecture enables zero-shot generalization to auxiliary tasks including vehicle color detection (89%), seatbelt detection (82%), and occupancy counting (78%) without task specific training. Through extensive experimentation on real world toll plaza imagery, we demonstrate that unified vision language approaches represent a paradigm shift in ALPR systems, offering superior accuracy, reduced architectural complexity, and emergent multi-task capabilities that traditional pipeline approaches cannot achieve.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
