相信模型：紧凑型 VLM 作为图像-文本数据质量的上下文内评判器
Trust the Model: Compact VLMs as In-Context Judges for Image-Text Data Quality
AI总结：
本文提出用专门微调的紧凑型 VLM 直接评判图像-文本样本质量并过滤噪声数据，以更低训练开销构建高质量多模态语料库。
摘要：
视觉-语言模型（VLM）通过整合视觉数据扩展了传统的大型语言模型，能够实现更丰富的多模态推理，并显著拓宽了人工智能的实际应用。然而，引入视觉输入也给维持数据质量带来了新的挑战。经验证据一致表明，与单纯增加数据数量相比，经过精心筛选且具有代表性的训练样本通常能够产生更优结果。受这一观察启发，我们提出一种精简的数据过滤框架，该框架采用在高质量图像-描述标注数据集上进行微调的紧凑型 VLM。该模型能够根据描述与图像的质量以及二者的对齐情况，有效评估并过滤潜在训练样本。以往方法通常在现有全规模 VLM 之上增加辅助过滤模块，与之不同，我们的方法专门利用一个专门构建的小型 VLM 的内在评估能力。该策略无需额外模块，并降低了训练开销。我们的轻量级模型能够高效过滤不准确、含噪的网络数据，改善图像-文本对齐和描述语言流畅度。实验结果表明，使用我们的紧凑型 VLM 进行高精度过滤后的数据集，其表现与通过大规模网络爬取收集的更大、更嘈杂的数据集相当，甚至更优。因此，我们的方法为构建高质量视觉-语言训练语料库提供了一种轻量而稳健的解决方案。可用性与实现：我们的紧凑型 VLM 过滤模型、训练数据、实用脚本和补充数据（附录）可在 https://github.com/daulettoibazar/Compact_VLM_Filter 免费获取。
英文摘要：
Vision-language models (VLMs) extend the conventional large language models by integrating visual data, enabling richer multimodal reasoning and significantly broadens the practical applications of AI. However, including visual inputs also brings new challenges in maintaining data quality. Empirical evidence consistently shows that carefully curated and representative training examples often yield superior results compared to simply increasing the quantity of data. Inspired by this observation, we introduce a streamlined data filtration framework that employs a compact VLM, fine-tuned on a high-quality image-caption annotated dataset. This model effectively evaluates and filters potential training samples based on caption and image quality and alignment. Unlike previous approaches, which typically add auxiliary filtration modules on top of existing full-scale VLMs, our method exclusively utilizes the inherent evaluative capability of a purpose-built small VLM. This strategy eliminates the need for extra modules and reduces training overhead. Our lightweight model efficiently filters out inaccurate, noisy web data, improving image-text alignment and caption linguistic fluency. Experimental results show that datasets underwent high-precision filtration using our compact VLM perform on par with, or even surpass, larger and noisier datasets gathered through high-volume web crawling. Thus, our method provides a lightweight yet robust solution for building high-quality vision-language training corpora. \\ \textbf{Availability and implementation:} Our compact VLM filtration model, training data, utility scripts, and Supplementary data (Appendices) are freely available at https://github.com/daulettoibazar/Compact_VLM_Filter.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
