发表机构
Macquarie University; York University; Northern Illinois University; University of Technology Sydney; North South University; Lancaster University; Australian National University（麦考瑞大学; 约克大学; 北伊利诺伊大学; 悉尼科技大学; 北南大学; 兰卡斯特大学; 澳大利亚国立大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对LVLMs现有模型编辑技术采用单一全局子空间无法适配多样幻觉模式的问题，提出无训练的实例级解耦子空间框架，经多基准实验验证其鲁棒性、通用性与效率。
AI 中文摘要
近期，大型视觉语言模型（LVLMs）通过将视觉编码器与大型语言模型（LLMs）集成，已实现强大的多模态推理能力，但生成文本常出现不准确描述视觉输入的幻觉问题，严重影响其可靠性。尽管微调可缓解该问题，但计算成本高且需要大量精心整理的数据集，因此无训练的替代方案颇具吸引力。其中，模型编辑方法比基于解码的方法更具前景：解码方法针对每个输入调整输出，但会引入计算开销和不稳定性；而模型编辑离线修改内部表示，提供更高效稳定的解决方案。然而，现有模型编辑技术通常依赖单一全局子空间纠正幻觉，对所有测试样本一视同仁，无法捕捉不同输入间多样的幻觉模式。为解决此局限，我们提出一种无训练幻觉缓解框架，用于推理时的动态实例级抑制。该方法首先构建一组解耦幻觉子空间，每个子空间隔离一种 distinct 幻觉模式；推理时，模型自适应计算反映每个输入与这些子空间关系的权重，引导动态组合的投影，选择性抑制最可能的幻觉方向，同时保留基于图像的语义。在多个视觉语言基准和LVLM家族上的大量实验表明，该方法实现了一致的性能提升，凸显其鲁棒性、通用性和效率。
英文摘要
Recent advances in large vision-language models (LVLMs) have enabled powerful multimodal reasoning by integrating visual encoders with large language models (LLMs). However, their reliability is frequently undermined by hallucinations, where generated text inaccurately describes the visual input. Although fine-tuning can mitigate this problem, it is computationally expensive and requires large, curated datasets, making training-free alternatives attractive. Among these, model editing is more promising than decoding-based approaches: decoding methods adapt outputs per input but introduce computational overhead and instability, whereas model editing modifies internal representations offline, providing a more efficient and stable solution. However, existing model-editing techniques typically rely on a single global subspace to correct hallucinations, treating all test samples identically and failing to capture diverse hallucination modes across inputs. To address this limitation, we propose a training-free hallucination mitigation framework for dynamic, per-instance suppression at test time. Our method first constructs a set of Disentangled Hallucination Subspaces, each isolating a distinct hallucination mode. During inference, the model adaptively calculates weights reflecting each input's relationship to these subspaces, guiding a dynamically combined projection that selectively suppresses the most probable hallucination directions while preserving image-grounded semantics. Extensive experiments across multiple vision-language benchmarks and LVLM families demonstrate consistent improvements, highlighting the robustness, generalizability, and efficiency of our approach.
CommentsBMVC 2026
