BIMA：面向大型视觉语言模型幻觉预测与缓解的双射最大似然学习方法
BIMA: Bijective Maximum Likelihood Learning Approach to Hallucination Prediction and Mitigation in Large Vision-Language Models
AI总结：
本文提出基于归一化流理论的双射最大似然学习方法BIMA，通过改进解码过程有效缓解大型视觉语言模型的幻觉问题，在POPE基准上F1达85.06%，CHAIRS和CHAIRI分别降低7.6%和2.6%。
摘要：
大型视觉语言模型已被广泛采用以推动各个领域的发展。然而，开发具有最小可解释特征的大规模可信系统面临着重大挑战。与这些系统产生的错误功能相关的一个最普遍术语是幻觉，即语言模型生成与视觉内容不符的响应。为了缓解这一问题，目前已开发出多种方法，其中一个突出方向是改进解码过程。在本文中，我们提出了一种新的双射最大似然学习（BIMA）方法，利用归一化流理论来缓解幻觉。所提出的BIMA方法能有效缓解主流视觉语言模型中的幻觉问题，带来显著改善。值得注意的是，BIMA在POPE基准上取得了85.06%的平均F1分数，并显著将CHAIRS和CHAIRI分别降低了7.6%和2.6%。据我们所知，这是首批探索利用双射手段来减少大型视觉语言模型引发幻觉的研究之一。
英文摘要：
Large vision-language models have become widely adopted to advance in various domains. However, developing a trustworthy system with minimal interpretable characteristics of large-scale models presents a significant challenge. One of the most prevalent terms associated with the fallacy functions caused by these systems is hallucination, where the language model generates a response that does not correspond to the visual content. To mitigate this problem, several approaches have been developed, and one prominent direction is to ameliorate the decoding process. In this paper, we propose a new Bijective Maximum Likelihood Learning (BIMA) approach to hallucination mitigation using normalizing flow theories. The proposed BIMA method can efficiently mitigate the hallucination problem in prevailing vision-language models, resulting in significant improvements. Notably, BIMA achieves the average F1 score of 85.06% on POPE benchmark and remarkably reduce CHAIRS and CHAIRI by 7.6% and 2.6%, respectively. To the best of our knowledge, this is one of the first studies that contemplates the bijection means to reduce hallucination induced by large vision-language models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
