配置上下文学习演示以释放多模态大语言模型情感感知能力的实证研究
An Empirical Study on Configuring In-Context Learning Demonstrations for Unleashing MLLMs' Sentimental Perception Capability
AI总结：
本研究通过扩展零样本范式至上下文学习并优化演示的检索、呈现和分布三个关键因素，验证了多模态大语言模型具备情感感知能力，并在六个数据集上分别提升平均准确率15.9%和11.2%。
摘要：
多模态大语言模型（MLLMs）的进步使得各种多模态任务能够在零样本范式下得到解决。该范式规避了模型微调的成本，成为实际应用中的主导趋势。然而，多模态情感分析（MSA）作为追求通用人工智能过程中的关键挑战，却无法适应这种便利性。零样本范式在MSA上表现出不理想的性能，这引发了人们对MLLMs能否像监督模型一样胜任地感知情感的质疑。通过将零样本范式扩展至上下文学习（ICL），并对演示配置进行深入研究，我们验证了MLLMs确实具备这种能力。具体而言，我们全面调查并优化了涵盖演示检索、呈现和分布的三个关键因素。我们还发现并有效抵消了MLLMs固有的情感预测偏差。通过三者互补，所设计的策略在六个MSA数据集上相较于零样本范式平均准确率提升了15.9%，相较于随机ICL基线提升了11.2%。
英文摘要：
The advancements in Multimodal Large Language Models (MLLMs) have enabled various multimodal tasks to be addressed under a zero-shot paradigm. This paradigm sidesteps the cost of model fine-tuning, emerging as a dominant trend in practical application. Nevertheless, Multimodal Sentiment Analysis (MSA), a pivotal challenge in the quest for general artificial intelligence, fails to accommodate this convenience. The zero-shot paradigm exhibits undesirable performance on MSA, casting doubt on whether MLLMs can perceive sentiments as competent as supervised models. By extending the zero-shot paradigm to In-Context Learning (ICL) and conducting an in-depth study on configuring demonstrations, we validate that MLLMs indeed possess such capability. Specifically, three key factors that cover demonstrations' retrieval, presentation, and distribution are comprehensively investigated and optimized. A sentimental predictive bias inherent in MLLMs is also discovered and later effectively counteracted. By complementing each other, the devised strategies for three factors result in average accuracy improvements of 15.9% on six MSA datasets against the zero-shot paradigm and 11.2% against the random ICL baseline.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
