发表机构
Australian National University; The University of New South Wales; University of Technology Sydney; York University; Lancaster University（澳大利亚国立大学; 新南威尔士大学; 悉尼科技大学; 约克大学; 兰卡斯特大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对大型视觉语言模型的物体幻觉问题，提出无训练的测试时幻觉缓解（TTH）方法，通过令牌验证器模块等技术提升模型准确性与稳健性，通用性和实用性良好。
AI 中文摘要
大型视觉语言模型（LVLMs）存在物体幻觉问题，即模型生成与输入图像不符的非事实内容，这仍是其在实际应用中可靠性的关键障碍。现有缓解策略可分为基于训练的方法和无训练方法：基于训练的方法通常性能较强，但成本高昂，需要大量计算资源、大规模数据和耗时的微调；无训练方法因效率高而颇具吸引力，但现有无训练方法要么需要多轮解码，增加了计算开销，要么以特定于模型的方式修改内部状态，存在降低预训练知识的风险。我们提出了测试时幻觉缓解（TTH）方法，一种新型无训练方法，可解决上述两个局限。TTH引入了一个令牌验证器模块，实现为零样本多模态分类器（MMC），以生成基于输入图像的辅助逻辑值；这些逻辑值在令牌级别与原始LVLM输出融合，融合对象为候选池中的对象令牌，随后应用基于熵的加权方案以实现稳健且准确的预测。在多个LVLM家族和不同基准上的大量实验表明，TTH可持续提升准确性和稳健性，凸显其通用性和实际有效性。代码已发布在该https URL。
英文摘要
Object Hallucination in large vision-language models (LVLMs), where models generate non-factual content about input images, remains a critical barrier to their reliability in real-world applications. Existing mitigation strategies can be categorized into training-based and training-free methods. Training-based methods often achieve strong performance but are costly, requiring extensive computational resources, large-scale data, and time-consuming fine-tuning. Training-free approaches are particularly appealing due to their efficiency. However, existing training-free methods either require multiple decoding rounds, which adds computational overhead, or modify internal states in a model-specific way that risks degrading pretrained knowledge. We propose Test-Time Hallucination Mitigation (TTH) method, a novel training-free method that addresses both limitations. TTH introduces a token-validator module, implemented as a zero-shot Multi-Modal Classifier (MMC), to generate auxiliary logits grounded in the input image. These logits are fused with the original LVLM outputs at the token level for object tokens selected from a candidate pool. An entropy-based weighting scheme is then applied to enable robust and accurate predictions. Extensive experiments across multiple LVLM families and diverse benchmarks demonstrate that TTH consistently improves accuracy and robustness, underscoring its generalizability and practical effectiveness. Code is released at https://github.com/Mehran-TAM/TTH
CommentsAccepted at ECCV 2026 MUCG
