TCAP: 面向MLLM微调中无监督后门检测的三组件注意力分析
TCAP: Tri-Component Attention Profiling for Unsupervised Backdoor Detection in MLLM Fine-Tuning
AI总结：
针对微调即服务中的后门攻击，提出基于三组件注意力分布差异的无监督防御框架TCAP，通过高斯混合模型和EM投票聚合检测中毒样本。
摘要：
微调即服务（FTaaS）促进了多模态大语言模型（MLLMs）的定制化，但通过中毒数据引入了严重的后门风险。现有防御要么依赖监督信号，要么无法泛化到多样的触发器类型和模态。在这项工作中，我们揭示了一个通用的后门指纹——注意力分配差异，即中毒样本破坏了系统指令、视觉输入和用户文本查询三个功能组件之间的平衡注意力分布，无论触发器形态如何。受此启发，我们提出三组件注意力分析（TCAP），一种无监督防御框架，用于过滤后门样本。TCAP将跨模态注意力图分解为三个组件，通过高斯混合模型（GMM）统计分析识别对触发器敏感的注意力头，并通过基于EM的投票聚合隔离中毒样本。跨多种MLLM架构和攻击方法的大量实验表明，TCAP实现了持续强劲的性能，使其成为MLLMs中稳健且实用的后门防御方法。
英文摘要：
Fine-Tuning-as-a-Service (FTaaS) facilitates the customization of Multimodal Large Language Models (MLLMs) but introduces critical backdoor risks via poisoned data. Existing defenses either rely on supervised signals or fail to generalize across diverse trigger types and modalities. In this work, we uncover a universal backdoor fingerprint-attention allocation divergence-where poisoned samples disrupt the balanced attention distribution across three functional components: system instructions, vision inputs, and user textual queries, regardless of trigger morphology. Motivated by this insight, we propose Tri-Component Attention Profiling (TCAP), an unsupervised defense framework to filter backdoor samples. TCAP decomposes cross-modal attention maps into the three components, identifies trigger-responsive attention heads via Gaussian Mixture Model (GMM) statistical profiling, and isolates poisoned samples through EM-based vote aggregation. Extensive experiments across diverse MLLM architectures and attack methods demonstrate that TCAP achieves consistently strong performance, establishing it as a robust and practical backdoor defense in MLLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
