未接地的接地:一种基于谱-图框架的多模态大语言模型幻觉量化方法
Grounding the Ungrounded: A Spectral-Graph Framework for Quantifying Hallucinations in Multimodal LLMs
AI总结:
本文提出了一种基于谱-图框架的多模态大语言模型幻觉量化方法,通过谱分解和RKHS本征模式,提供可解释的语义失真度量。
摘要:
在大语言模型中,尤其是多模态设置中,幻觉问题会损害可靠性。我们提出了一种严格的信息几何框架,基于扩散动力学,用于量化多模态大语言模型(MLLMs)中的幻觉。其中,模型输出通过多模态图拉普拉斯算子的谱分解进行嵌入,其与真实流形的差距定义了语义失真度量。我们推导了温度依赖的幻觉特征的Courant-Fischer界限,并利用RKHS本征模式获得具有模态意识的可解释度量,以跟踪提示和时间的演变。这将幻觉重新定义为可量化和有界的,为评估和缓解提供了系统的基础。
英文摘要:
Hallucinations in LLMs--especially in multimodal settings--undermine reliability. We present a rigorous information-geometric framework, grounded in diffusion dynamics, to quantify hallucinations in MLLMs where model outputs are embedded via spectral decompositions of multimodal graph Laplacians, and their gaps to a truth manifold define a semantic distortion metric. We derive Courant-Fischer bounds on a temperature-dependent hallucination profile and use RKHS eigenmodes to obtain modality-aware, interpretable measures that track evolution over prompts and time. This reframes hallucination as quantifiable and bounded, providing a principled basis for evaluation and mitigation.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。