VACoDe: Visual Augmented Contrastive Decoding（视觉增强对比解码）
VACoDe: Visual Augmented Contrastive Decoding
AI总结：
针对大视觉语言模型生成不准确的问题，提出VACoDe方法，通过softmax距离度量自适应选择最优图像增强方式进行对比解码，无需额外训练即可提升多类视觉-语言任务的输出质量，适配不同模型。
摘要：
尽管近期的大视觉语言模型（LVLM）表现惊人，但这类模型往往会生成不准确的响应。为解决该问题，过往研究聚焦于通过结合增强图像的对比解码（CD）来缓解幻觉问题，该方法会放大与原始图像的对比效果。但这类方法存在局限性：依赖单一增强方式，对部分任务而言限制性较强，且使用外部知识的成本较高。\n本研究通过探索如何利用多种图像增强方式来解决上述局限。经过大量实验，我们观察到不同的增强方式会根据任务类型产生不同程度的对比效果。基于这一观察，我们提出了名为VACoDe（Visual Augmented Contrastive Decoding，视觉增强对比解码）的新方法。该方法借助提出的softmax距离度量，为每个任务自适应选择对比效果最强的增强方式。\n我们的实证测试表明，VACoDe的表现优于过往方法，在各类视觉-语言任务中提升了输出质量。此外，VACoDe可普遍应用于不同类型和规模的模型，无需额外训练，也无需使用外部模型和数据。
英文摘要：
Despite the astonishing performance of recent Large Vision-Language Models (LVLMs), these models often generate inaccurate responses. To address this issue, previous studies have focused on mitigating hallucinations by employing contrastive decoding (CD) with augmented images, which amplifies the contrast with the original image. However, these methods have limitations, including reliance on a single augmentation, which is restrictive for certain tasks, as well as the high cost of using external knowledge. In this study, we address these limitations by exploring how to utilize multiple image augmentations. Through extensive experiments, we observed that different augmentations produce varying levels of contrast depending on the task. Based on this observation, we introduce a novel method called VACoDe, Visual Augmented Contrastive Decoding. This method adaptively selects the augmentation with the highest contrast for each task using the proposed softmax distance metric. Our empirical tests show that \alg outperforms previous methods and improves output quality in various vision-language tasks. Additionally, VACoDe can be universally applied across different model types and sizes without additional training or the use of external models and data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
