当大模型训练小模型：利用小型视觉语言模型实现高效视觉问答的无标签模型对齐
When Big Models Train Small Ones: Label-Free Model Parity Alignment for Efficient Visual Question Answering using Small VLMs
AI总结：
本文提出模型对齐器MPA，通过无标签图像和基于差异对齐的知识迁移策略，在不依赖标注数据的情况下系统提升小型视觉语言模型在多个VQA基准上的性能，缩小与大型模型的差距并保持计算效率。
摘要：
大型视觉语言模型（L-VLMs）在包括视觉问答（VQA）在内的各种视觉和语言任务中展现了卓越的性能。然而，它们高昂的计算成本使其在资源受限环境和推理密集型应用中不切实际。相比之下，小型视觉语言模型（S-VLMs）虽然具有效率优势，但与大型对应模型相比存在显著的性能差距。在本工作中，我们提出了模型对齐器（Model Parity Aligner，MPA），这是一个新颖的框架，旨在通过利用无标签图像和来自L-VLMs的有效知识迁移来系统性地提升S-VLMs。与依赖有标签训练数据的传统知识蒸馏方法不同，MPA采用了一种基于策略的对齐方法，能够精确识别S-VLMs与L-VLMs之间的知识差异，并仅针对这些差异优化训练。我们在四个多样化的VQA基准上进行了广泛实验，即TextVQA、ST-VQA、ChartQA和OKVQA，每个基准都需要专门的推理能力，如文本识别、图表解释以及常识和事实理解。我们的结果表明，MPA在所有基准上一致地提升了S-VLMs的性能，在保持计算效率的同时缩小了性能差距。我们已将代码公开。
英文摘要：
Large Vision-Language Models (L-VLMs) have demonstrated remarkable performance in various vision and language tasks, including visual question answering (VQA). However, their high computational cost makes them impractical for resource-constrained settings and inference-heavy applications. In contrast, Small Vision-Language Models (S-VLMs) offer efficiency but suffer from a significant performance gap compared to their larger counterparts. In this work, we introduce the Model Parity Aligner (MPA), a novel framework designed to systematically improve S-VLMs by leveraging unlabeled images and effective knowledge transfer from L-VLMs. Instead of traditional knowledge distillation methods that rely on labeled training data, MPA employs a strategic parity-based approach that precisely identifies the knowledge disparities between S-VLMs and L-VLMs, and optimizes training by targeting only these disparities. We conduct extensive experiments on four diverse VQA benchmarks, namely TextVQA, ST-VQA, ChartQA, and OKVQA, each of which requires specialized reasoning capabilities such as text recognition, chart interpretation, and commonsense and factual understanding. Our results demonstrate that MPA consistently enhances the performance of S-VLMs on all benchmarks, reducing the performance gap while maintaining computational efficiency. We make our code publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
