Metropolis-Hastings描述游戏：通过去中心化贝叶斯推理融合视觉语言模型的知识
Metropolis-Hastings Captioning Game: Knowledge Fusion of Vision Language Models via Decentralized Bayesian Inference
AI总结：
针对现有多视觉语言模型（VLMs）组合方法存在的推理成本和架构约束问题，提出Metropolis-Hastings描述游戏（MHCG），通过去中心化贝叶斯推理实现多VLMs知识融合，在图像描述任务中提升了无参考评估指标并促进了词汇共享。
摘要：
我们提出Metropolis-Hastings描述游戏（MHCG），这是一种通过相互学习融合多个视觉语言模型（VLMs）知识的方法。尽管现有组合多模型的方法存在推理成本高和架构约束的问题，但MHCG通过类似语言游戏的过程执行去中心化贝叶斯推理，从而避免了这些问题。知识融合过程建立两个VLM智能体之间的通信，它们交替对图像进行描述并相互学习。我们使用两个分别在不同数据集上预训练的VLMs进行了两项图像描述实验。第一项实验表明，MHCG在无参考评估指标上实现了持续改进；第二项实验通过观察生成描述中词汇的出现情况，研究了MHCG如何促进VLMs类别级词汇的共享。
英文摘要：
We propose the Metropolis-Hastings Captioning Game (MHCG), a method to fuse knowledge of multiple vision-language models (VLMs) by learning from each other. Although existing methods that combine multiple models suffer from inference costs and architectural constraints, MHCG avoids these problems by performing decentralized Bayesian inference through a process resembling a language game. The knowledge fusion process establishes communication between two VLM agents alternately captioning images and learning from each other. We conduct two image-captioning experiments with two VLMs, each pre-trained on a different dataset. The first experiment demonstrates that MHCG achieves consistent improvement in reference-free evaluation metrics. The second experiment investigates how MHCG contributes to sharing VLMs' category-level vocabulary by observing the occurrence of the vocabulary in the generated captions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
