MLLM提示工程的未来是自适应的：面向鲁棒多模态性能的提示工程方法综合实验评估
The Future of MLLM Prompting is Adaptive: A Comprehensive Experimental Evaluation of Prompt Engineering Methods for Robust Multimodal Performance
AI总结：
针对MLLMs提示工程优化问题，本文综合评估7种提示方法在13个开源MLLMs的24项任务上的表现，发现无单一最优方法，提出结合示例引导与选择性结构化推理的自适应策略可提升鲁棒性与准确性。
摘要：
多模态大型语言模型（MLLMs）通过整合文本、图像和代码等多种模态，有望改变机器处理和生成类人响应的方式。然而，有效利用其能力取决于优化的提示工程。我们对7种提示工程方法进行了综合实验评估，这些方法应用于13个开源MLLMs，涵盖24项任务，包括推理与组合性、多模态理解与对齐、复杂代码生成与执行、知识检索与整合。我们的方法按参数数量将模型分为小型（<4B）、中型（4B-10B）和大型（>10B）三类，并比较了零样本、单样本、少样本、思维链、类比、生成知识和思维树等提示技术。大型MLLMs在代码生成等结构化任务中表现出色，在少样本提示下准确率高达96.88%，但所有模型在复杂推理和抽象理解任务中均存在困难，准确率常低于60%且幻觉率较高。结构化推理提示常使小型模型幻觉率升至75%，并导致大型模型响应时间延长（超过20秒），而更简单的提示方法则提供了更简洁高效的输出。没有单一提示方法能统一优化所有任务类型，相反，结合基于示例的引导与选择性结构化推理的自适应策略对于提升鲁棒性、效率和事实准确性至关重要。我们的研究结果为提示工程提供了实用建议，支持MLLMs在AI辅助编码、知识检索和多模态内容理解等应用中更可靠地部署。
英文摘要：
Multimodal Large Language Models (MLLMs) are set to transform how machines process and generate human-like responses by integrating diverse modalities such as text, images, and code. Yet, effectively harnessing their capabilities hinges on optimal prompt engineering. We present a comprehensive experimental evaluation of seven prompt engineering methods applied to 13 open-source MLLMs over 24 tasks spanning Reasoning and Compositionality, Multimodal Understanding and Alignment, Complex Code Generation and Execution, and Knowledge Retrieval and Integration. Our approach stratifies models by parameter count into Small (<4B), Medium (4B-10B), and Large (>10B) categories and compares prompting techniques including Zero-Shot, One-Shot, Few-Shot, Chain-of-Thought, Analogical, Generated Knowledge, and Tree-of-Thought. While Large MLLMs excel in structured tasks such as code generation, achieving accuracies up to 96.88% under Few-Shot prompting, all models struggle with complex reasoning and abstract understanding, often yielding accuracies below 60% and high hallucination rates. Structured reasoning prompts frequently increased hallucination up to 75% in small models and led to longer response times (over 20 seconds in Large MLLMs), while simpler prompting methods provided more concise and efficient outputs. No single prompting method uniformly optimises all task types. Instead, adaptive strategies combining example-based guidance with selective structured reasoning are essential to enhance robustness, efficiency, and factual accuracy. Our findings offer practical recommendations for prompt engineering and support more reliable deployment of MLLMs across applications including AI-assisted coding, knowledge retrieval, and multimodal content understanding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
