AG-VAS：基于大多模态模型的锚引导零样本视觉异常分割
AG-VAS: Anchor-Guided Zero-Shot Visual Anomaly Segmentation with Large Multimodal Models
AI总结：
AG-VAS通过引入三个可学习的语义锚点，建立统一的锚引导分割范式，提升零样本视觉异常分割的精度与性能。
摘要：
大多模态模型（LMMs）展现出强大的任务泛化能力，为零样本视觉异常分割（ZSAS）提供了新机遇。然而，现有基于LMM的分割方法仍面临根本性限制：异常概念本质上抽象且依赖上下文，缺乏稳定的视觉原型，且高层语义嵌入与像素级空间特征之间的弱对齐阻碍了精确的异常定位。为解决这些挑战，我们提出了AG-VAS（锚引导视觉异常分割），一种新的框架，通过扩展LMM词汇表，引入三个可学习的语义锚点-[SEG]、[NOR]和[ANO]，建立统一的锚引导分割范式。具体而言，[SEG]作为绝对语义锚点，将抽象的异常语义转化为显式的、空间上扎根的视觉实体（例如孔洞或划痕），而[NOR]和[ANO]作为相对锚点，建模不同类别之间正常与异常模式的上下文对比。为进一步增强跨模态对齐，我们引入了语义-像素对齐模块（SPAM），将语言层面的语义嵌入与高分辨率视觉特征对齐，并引入了锚引导的掩码解码器（AGMD），进行锚点条件的掩码预测以实现精确的异常定位。此外，我们构建了Anomaly-Instruct20K，一个大规模指令数据集，将异常知识组织为结构化的外观、形状和空间属性描述，促进有效学习和整合所提出的语义锚点。在六个工业和医疗基准上的广泛实验表明，AG-VAS在零样本设置中实现了稳定的最佳性能。
英文摘要：
Large multimodal models (LMMs) exhibit strong task generalization capabilities, offering new opportunities for zero-shot visual anomaly segmentation (ZSAS). However, existing LMM-based segmentation approaches still face fundamental limitations: anomaly concepts are inherently abstract and context-dependent, lacking stable visual prototypes, and the weak alignment between high-level semantic embeddings and pixel-level spatial features hinders precise anomaly localization. To address these challenges, we present AG-VAS (Anchor-Guided Visual Anomaly Segmentation), a new framework that expands the LMM vocabulary with three learnable semantic anchor tokens-[SEG], [NOR], and [ANO], establishing a unified anchor-guided segmentation paradigm. Specifically, [SEG] serves as an absolute semantic anchor that translates abstract anomaly semantics into explicit, spatially grounded visual entities (e.g., holes or scratches), while [NOR] and [ANO] act as relative anchors that model the contextual contrast between normal and abnormal patterns across categories. To further enhance cross-modal alignment, we introduce a Semantic-Pixel Alignment Module (SPAM) that aligns language-level semantic embeddings with high-resolution visual features, along with an Anchor-Guided Mask Decoder (AGMD) that performs anchor-conditioned mask prediction for precise anomaly localization. In addition, we curate Anomaly-Instruct20K, a large-scale instruction dataset that organizes anomaly knowledge into structured descriptions of appearance, shape, and spatial attributes, facilitating effective learning and integration of the proposed semantic anchors. Extensive experiments on six industrial and medical benchmarks demonstrate that AG-VAS achieves consistent state-of-the-art performance in the zero-shot setting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
