见、解释与干预：一种少样本多模态代理框架用于仇恨表情包审查
PEST: Parameter Efficient Steering of Blackbox VLMs via Agentic Few-shot Alignment for Hateful Meme Moderation
AI总结：
本文提出了一种少样本多模态代理框架，用于在有限数据条件下实现通用仇恨表情包审查，通过检测、解释和干预三方面提升审查效果。
摘要：
在本文中，我们从三个互补的角度审视仇恨表情包——如何检测它们、如何解释其内容以及如何在发布前进行干预——通过在生成式AI模型上应用一系列策略。据我们所知，解释和干预通常与检测分开研究，这并不反映现实世界的情况。此外，由于为表情包审查编纂大型标注数据集成本高昂，我们提出了一种新的框架，利用任务特定的生成式多模态代理和大多模态模型的少样本适应性，以应对不同类型的表情包。我们认为这是首次在有限数据条件下专注于通用仇恨表情包审查的工作，并在现实世界生产场景中具有很强的部署潜力。警告：包含可能有毒的内容。
英文摘要：
In this work, we examine hateful memes from three complementary angles - how to detect them, how to explain their content and how to intervene them before being posted - by applying a range of strategies built on top of generative AI models. To the best of our knowledge, explanation and intervention have typically been studied separately from detection, which does not reflect real-world conditions. Further, since curating large annotated datasets for meme moderation is prohibitively expensive, we propose a novel framework - PEST - that leverages task-specific generative VLMs and the few-shot adaptability of large VLMs to cater to different types of memes. We believe this is the first work focused on generalizable hateful meme moderation under limited data conditions, and has strong potential for deployment in real-world production scenarios. Warning: Contains potentially toxic contents.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
