多模态大语言模型中的视觉提示：一项综述
Visual Prompting in Multimodal Large Language Models: A Survey
AI总结：
本文综述了多模态大语言模型中的视觉提示方法，涵盖提示分类、生成、组合推理及提示学习，并总结了提升模型对齐与感知的训练方法及未来展望。
摘要：
多模态大语言模型（MLLMs）为预训练大语言模型（LLMs）配备了视觉能力。尽管LLMs中的文本提示已被广泛研究，但视觉提示作为更细粒度、更自由形式的视觉指令开始兴起。本文提出了关于MLLMs中视觉提示方法的首篇全面综述，重点关注视觉提示、提示生成、组合推理和提示学习。我们对现有视觉提示进行分类，并讨论了在图像上自动进行提示标注的生成方法。我们还考察了能够实现视觉编码器与骨干LLMs更好对齐的视觉提示方法，涉及MLLM的视觉定位、对象指代和组合推理能力。此外，我们总结了模型训练和上下文学习方法，以提升MLLM对视觉提示的感知与理解。本文审视了MLLMs中开发的视觉提示方法，并对这些方法的未来提供了展望。
英文摘要：
Multimodal large language models (MLLMs) equip pre-trained large-language models (LLMs) with visual capabilities. While textual prompting in LLMs has been widely studied, visual prompting has emerged for more fine-grained and free-form visual instructions. This paper presents the first comprehensive survey on visual prompting methods in MLLMs, focusing on visual prompting, prompt generation, compositional reasoning, and prompt learning. We categorize existing visual prompts and discuss generative methods for automatic prompt annotations on the images. We also examine visual prompting methods that enable better alignment between visual encoders and backbone LLMs, concerning MLLM's visual grounding, object referring, and compositional reasoning abilities. In addition, we provide a summary of model training and in-context learning methods to improve MLLM's perception and understanding of visual prompts. This paper examines visual prompting methods developed in MLLMs and provides a vision of the future of these methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
