FBHM：用于仇恨模因检测的功能性基准测试与视觉语言模型引导
FBHM: Functional Benchmarking and Steering of VLMs for Hateful Meme Detection
AI总结：
针对现有基准无法因果评估视觉语言模型漏洞的问题，提出基于25种修辞功能和10个目标社区构建的FBHM基准，并采用可学习引导向量（LSV）在极低数据量下提升模型性能约30个Macro-F1点。
摘要：
仇恨模因检测对于视觉语言模型仍是一个严峻挑战，因为现有基准在结构上是观察性的——混淆了修辞仇恨机制与目标社区特征，并阻碍了对模型漏洞的因果评估。为解决这一问题，我们引入了FBHM，一个系统策划的基于功能的仇恨模因基准，沿两个正交轴构建：25种不同的修辞功能和10个目标社区（总共5,000个模因）。对最先进的视觉语言模型进行基准测试揭示了一个严重的泛化差距：在标准数据集上高度准确的模型在FBHM上灾难性地下降到接近随机性能，证明它们利用了数据集特定的启发式方法而非稳健的多模态推理。为了高效缩小这一差距，我们提出了LSV（可学习引导向量），一种超低数据量策略，在仅500个引导样本（50个独特基础模因）上应用因果干预目标，将FBHM性能提升约30个Macro-F1点，同时优于上下文学习和PEFT，且不降低源域性能。
英文摘要：
Hateful meme detection remains a formidable challenge for vision-language models, as existing benchmarks are structurally observational - confounding rhetorical hate mechanisms with target community features and preventing causal evaluation of model vulnerabilities. To address this, we introduce FBHM, a systematically curated benchmark of Functionality Based Hateful Memes constructed along two orthogonal axes: 25 distinct rhetorical functionalities and 10 target communities (5,000 memes total). Benchmarking state-of-the-art VLMs reveals a severe generalization gap: models highly accurate on standard datasets catastrophically drop to near-random performance on FBHM, proving they exploit dataset-specific heuristics rather than robust multimodal reasoning. To efficiently close this gap, we propose LSV (learnable steering vectors), an ultra-low data regime strategy that applies a causal intervention objective on as few as 500 steering samples (50 unique base memes), boosting FBHM performance by ~30 Macro-F1 points while outperforming in-context learning and PEFT without degrading source-domain performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
