少样本学习视角下多模态大语言模型的语言能力评估
Evaluating Linguistic Capabilities of Multimodal LLMs in the Lens of Few-Shot Learning
AI总结：
本研究在VALSE基准上评估多模态大语言模型的语言能力，发现少样本上下文学习和思维链提示显著提升推理性能，且预训练数据组成影响模型表现。
摘要：
多模态大语言模型（MLLMs）的语言能力对其在不同任务中的有效应用至关重要。本研究旨在评估MLLMs在VALSE基准上的表现，重点关注少样本上下文学习（ICL）和思维链（CoT）提示的有效性。我们对多种最先进的MLLMs进行了全面评估，这些模型在模型规模和预训练数据集上有所不同。实验结果表明，ICL和CoT提示显著提升了模型性能，尤其是在需要复杂推理和上下文理解的任务中。在字幕数据集上预训练的模型表现出优越的零样本性能，而在交错图像-文本数据上训练的模型则受益于少样本学习。我们的研究结果为优化MLLMs以更好地将语言与视觉上下文对齐提供了有价值的见解，强调了预训练数据组成的重要性以及少样本学习策略在提升MLLMs推理能力方面的潜力。
英文摘要：
The linguistic capabilities of Multimodal Large Language Models (MLLMs) are critical for their effective application across diverse tasks. This study aims to evaluate the performance of MLLMs on the VALSE benchmark, focusing on the efficacy of few-shot In-Context Learning (ICL), and Chain-of-Thought (CoT) prompting. We conducted a comprehensive assessment of state-of-the-art MLLMs, varying in model size and pretraining datasets. The experimental results reveal that ICL and CoT prompting significantly boost model performance, particularly in tasks requiring complex reasoning and contextual understanding. Models pretrained on captioning datasets show superior zero-shot performance, while those trained on interleaved image-text data benefit from few-shot learning. Our findings provide valuable insights into optimizing MLLMs for better grounding of language in visual contexts, highlighting the importance of the composition of pretraining data and the potential of few-shot learning strategies to improve the reasoning abilities of MLLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
