VORD：用于缓解大型视觉语言模型中物体幻觉的视觉序数校准
VORD: Visual Ordinal Calibration for Mitigating Object Hallucinations in Large Vision-Language Models
AI总结：
针对大型视觉语言模型的物体幻觉问题，提出VORD方法，通过修改图像对间的序数关系校准token预测，包含无训练和可训练两种形式，有效缓解幻觉并提升校准效果。
摘要：
大型视觉语言模型（LVLMs）随着大型语言模型的近期激增取得了显著进展。尽管取得了进步，LVLMs仍倾向于基于提供的源内容生成看似合理但不准确或不一致的信息。这种现象也被称为“幻觉”，在LVLMs部署期间可能产生严重的下游影响。为了解决这一问题，我们提出了VORD，一种简单有效的方法，通过基于修改图像对之间的序数关系校准token预测来缓解幻觉。VORD以两种形式呈现：1）一种极简的无训练变体，从修改图像对中消除不合理token；2）一种可训练目标函数，惩罚不可能的token。我们的实验表明，VORD提供了更好的校准，并在广泛的LVLM基准上有效缓解了物体幻觉。
英文摘要：
Large Vision-Language Models (LVLMs) have made remarkable developments along with the recent surge of large language models. Despite their advancements, LVLMs have a tendency to generate plausible yet inaccurate or inconsistent information based on the provided source content. This phenomenon, also known as ``hallucinations" can have serious downstream implications during the deployment of LVLMs. To address this, we present VORD a simple and effective method that alleviates hallucinations by calibrating token predictions based on ordinal relationships between modified image pairs. VORD is presented in two forms: 1.) a minimalist training-free variant which eliminates implausible tokens from modified image pairs, and 2.) a trainable objective function that penalizes unlikely tokens. Our experiments demonstrate that VORD delivers better calibration and effectively mitigates object hallucinations on a wide-range of LVLM benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
