少说多互动：评估多模态大语言模型中的上下文对话适应能力
Talk Less, Interact Better: Evaluating In-context Conversational Adaptation in Multimodal LLMs
AI总结：
本文提出自动化框架 ICCA，用于评估多模态大语言模型在对话中自发提高语言效率的能力，发现现有 MLLMs 缺乏该能力，仅 GPT-4 等模型可通过强力提示激发。
摘要：
人类在互动过程中会自发地使用越来越高效的语言，通过适应并形成临时约定。这一现象已通过参照游戏得到广泛研究，展现出人类语言超越意图传递的特性。然而，多模态大语言模型（MLLMs）是否在互动中同样提高沟通效率，以及它们可能采用何种机制实现这一目标，仍未得到探索。我们提出 ICCA，一个用于评估 MLLMs 中此类对话适应作为上下文行为的自动化框架。我们评估了多个最先进的 MLLMs，并观察到：尽管它们可能理解对话者越来越高效的语言，但并不会自发地让自己的语言随时间变得更加高效。后一种能力仅能在部分模型（如 GPT-4）中通过强力提示被激发。这表明，尽管这种语言互动特性是人类语言的常见标志，但它并未从当前的训练机制中自然产生。ICCA 可在 https://github.com/lil-lab/ICCA 获取。
英文摘要：
Humans spontaneously use increasingly efficient language as interactions progress, by adapting and forming ad-hoc conventions. This phenomenon has been studied extensively using reference games, showing properties of human language that go beyond relaying intents. It remains unexplored whether multimodal large language models (MLLMs) similarly increase communication efficiency during interactions, and what mechanisms they may adopt for this purpose. We introduce ICCA, an automated framework to evaluate such conversational adaptation as an in-context behavior in MLLMs. We evaluate several state-of-the-art MLLMs, and observe that while they may understand the increasingly efficient language of their interlocutor, they do not spontaneously make their own language more efficient over time. This latter ability can only be elicited in some models (e.g., GPT-4) with heavy-handed prompting. This shows that this property of linguistic interaction does not arise from current training regimes, even though it is a common hallmark of human language. ICCA is available at https://github.com/lil-lab/ICCA.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
