arXiv 2601.105512026-01-15cs.CV

释放大型视觉-语言模型的能力以实现道路基础设施的智能感知

Unleashing the Capabilities of Large Vision-Language Models for Intelligent Perception of Roadside Infrastructure

State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS)（信息工程测绘遥感国家重点实验室）；Wuhan University（武汉大学）；Hubei Luojia Laboratory（湖北珞珈实验室）

Luxuan Fu, Chong Liu, Bisheng Yang, Zhen Dong

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出一种领域适应框架，通过数据高效微调和知识引导推理，提升大型视觉语言模型在城市道路基础设施感知中的性能和专业合规性。

摘要：

城市道路基础设施的自动化感知对于智慧城市管理至关重要，但通用模型往往难以捕捉必要的细粒度属性和领域规则。尽管大型视觉语言模型（VLMs）在开放世界识别方面表现优异，但它们在符合工程标准的情况下准确解释复杂设施状态的能力有限，导致在现实应用中的可靠性不足。为此，我们提出了一种领域适应框架，将VLMs转化为专门的智能基础设施分析代理。我们的方法整合了数据高效的微调策略与知识引导的推理机制。具体来说，我们利用Grounding DINO上的开放词汇微调来稳健地定位多种资产，仅需最小监督；随后在Qwen-VL上使用LoRA进行深度语义属性推理。为减轻幻觉并强制专业合规，我们引入了双模态检索增强生成（RAG）模块，在推理过程中动态检索权威行业标准和视觉范例。在全面的新城市道路场景数据集上评估，我们的框架实现了58.9 mAP的检测性能和95.5%的属性识别准确率，展示了智能基础设施监控的稳健解决方案。

英文摘要：

Automated perception of urban roadside infrastructure is crucial for smart city management, yet general-purpose models often struggle to capture the necessary fine-grained attributes and domain rules. While Large Vision Language Models (VLMs) excel at open-world recognition, they often struggle to accurately interpret complex facility states in compliance with engineering standards, leading to unreliable performance in real-world applications. To address this, we propose a domain-adapted framework that transforms VLMs into specialized agents for intelligent infrastructure analysis. Our approach integrates a data-efficient fine-tuning strategy with a knowledge-grounded reasoning mechanism. Specifically, we leverage open-vocabulary fine-tuning on Grounding DINO to robustly localize diverse assets with minimal supervision, followed by LoRA-based adaptation on Qwen-VL for deep semantic attribute reasoning. To mitigate hallucinations and enforce professional compliance, we introduce a dual-modality Retrieval-Augmented Generation (RAG) module that dynamically retrieves authoritative industry standards and visual exemplars during inference. Evaluated on a comprehensive new dataset of urban roadside scenes, our framework achieves a detection performance of 58.9 mAP and an attribute recognition accuracy of 95.5%, demonstrating a robust solution for intelligent infrastructure monitoring.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。