面向多样化工作场所安全合规性可解释细粒度检测的视觉语言模型
Vision Language Model for Interpretable and Fine-grained Detection of Safety Compliance in Diverse Workplaces
AI总结：
针对传统目标检测模型在多样化工作场所PPE细粒度合规检测上的局限，提出含四个模块的可解释VLM框架Clip2Safety，在6种真实场景中准确率超越现有最优问答式VLM，推理速度快200倍。
摘要：
因个人防护装备（PPE）佩戴不合规导致的工作场所事故引发了严重的安全问题，并会带来法律责任、经济处罚以及声誉损失。尽管目标检测模型已展现出通过识别安全装备来解决该问题的能力，但现有大多数模型（如YOLO、Faster R-CNN和SSD）在多样化工作场景中验证PPE的细粒度属性方面存在局限。视觉语言模型（VLM）凭借视觉与文本信息的协同作用，正逐渐在检测任务中受到关注，为传统目标检测在PPE识别方面的局限提供了有前景的解决方案。然而，由于工作环境的复杂性和多变性，VLM在持续验证PPE属性方面仍面临挑战，需要其同时解读特定上下文的语言和视觉线索。我们提出了Clip2Safety——一个面向多样化工作场所安全合规性的可解释检测框架，包含四个核心模块：场景识别、视觉提示、安全装备检测和细粒度验证。场景识别模块识别当前场景以确定所需的安全装备；视觉提示模块生成检测过程所需的特定视觉提示；安全装备检测模块根据指定场景识别人员是否佩戴了所需安全装备；最后，细粒度验证模块评估已佩戴的安全装备是否满足细粒度的属性要求。我们在六种不同场景下开展了真实案例研究，结果表明，Clip2Safety不仅相比基于问答的现有最优VLM实现了准确率提升，推理速度还快两百倍。
英文摘要：
Workplace accidents due to personal protective equipment (PPE) non-compliance raise serious safety concerns and lead to legal liabilities, financial penalties, and reputational damage. While object detection models have shown the capability to address this issue by identifying safety items, most existing models, such as YOLO, Faster R-CNN, and SSD, are limited in verifying the fine-grained attributes of PPE across diverse workplace scenarios. Vision language models (VLMs) are gaining traction for detection tasks by leveraging the synergy between visual and textual information, offering a promising solution to traditional object detection limitations in PPE recognition. Nonetheless, VLMs face challenges in consistently verifying PPE attributes due to the complexity and variability of workplace environments, requiring them to interpret context-specific language and visual cues simultaneously. We introduce Clip2Safety, an interpretable detection framework for diverse workplace safety compliance, which comprises four main modules: scene recognition, the visual prompt, safety items detection, and fine-grained verification. The scene recognition identifies the current scenario to determine the necessary safety gear. The visual prompt formulates the specific visual prompts needed for the detection process. The safety items detection identifies whether the required safety gear is being worn according to the specified scenario. Lastly, the fine-grained verification assesses whether the worn safety equipment meets the fine-grained attribute requirements. We conduct real-world case studies across six different scenarios. The results show that Clip2Safety not only demonstrates an accuracy improvement over state-of-the-art question-answering based VLMs but also achieves inference times two hundred times faster.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
