T2Vs 遇见 VLMs：用于视觉有害性识别的可扩展多模态数据集
T2Vs Meet VLMs: A Scalable Multimodal Dataset for Visual Harmfulness Recognition
AI总结：
本文构建涵盖10类、含10,000张图像与1,000个视频的 VHD11K，并提出多 VLM 辩论式 VQA 标注框架，以提升视觉有害内容识别的泛化与可靠性。
摘要：
为应对遭遇不当或有害内容的风险，研究人员设法将若干有害内容数据集与机器学习方法结合，以检测有害概念。然而，现有有害数据集依据范围狭窄的有害对象的存在来策划，并且只覆盖真实有害内容来源。这阻碍了基于此类数据集的方法的泛化能力，并可能导致误判。因此，我们提出一个全面的有害数据集 Visual Harmful Dataset 11K（VHD11K），由10,000张图像和1,000个视频组成，这些数据从互联网爬取并由4个生成模型生成，共涵盖10个有害类别，覆盖具有非平凡定义的完整有害概念谱系。我们还提出一种新颖的标注框架，将标注过程表述为多智能体视觉问答（VQA）任务，让3个不同的 VLM 就给定图像/视频是否有害进行“辩论”，并在辩论过程中引入上下文学习策略。因此，我们能够确保 VLM 在作出决策前彻底考虑给定图像/视频的上下文以及双方论点，进一步降低边缘情形下发生误判的可能性。评估和实验结果表明：（1）我们新颖标注框架产生的标注与人工标注高度一致，确保了 VHD11K 的可靠性；（2）我们的全谱系有害数据集成功识别出现有有害内容检测方法在检测广泛有害内容方面的无能，并提升了现有有害性识别方法的性能；（3）VHD11K 优于基线数据集 SMID，有害性识别方法上更优的提升证明了这一点。完整数据集和代码可在 https://github.com/nctu-eva-lab/VHD11K 获取。
英文摘要：
To address the risks of encountering inappropriate or harmful content, researchers managed to incorporate several harmful contents datasets with machine learning methods to detect harmful concepts. However, existing harmful datasets are curated by the presence of a narrow range of harmful objects, and only cover real harmful content sources. This hinders the generalizability of methods based on such datasets, potentially leading to misjudgments. Therefore, we propose a comprehensive harmful dataset, Visual Harmful Dataset 11K (VHD11K), consisting of 10,000 images and 1,000 videos, crawled from the Internet and generated by 4 generative models, across a total of 10 harmful categories covering a full spectrum of harmful concepts with nontrivial definition. We also propose a novel annotation framework by formulating the annotation process as a multi-agent Visual Question Answering (VQA) task, having 3 different VLMs "debate" about whether the given image/video is harmful, and incorporating the in-context learning strategy in the debating process. Therefore, we can ensure that the VLMs consider the context of the given image/video and both sides of the arguments thoroughly before making decisions, further reducing the likelihood of misjudgments in edge cases. Evaluation and experimental results demonstrate that (1) the great alignment between the annotation from our novel annotation framework and those from human, ensuring the reliability of VHD11K; (2) our full-spectrum harmful dataset successfully identifies the inability of existing harmful content detection methods to detect extensive harmful contents and improves the performance of existing harmfulness recognition methods; (3) VHD11K outperforms the baseline dataset, SMID, as evidenced by the superior improvement in harmfulness recognition methods. The complete dataset and code can be found at https://github.com/nctu-eva-lab/VHD11K.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
