需要多少反事实？通过电路和因果效应探究VLM幻觉
How Many Counterfactuals Does It Take? Probing VLM Hallucinations Through Circuits and Causal Effects
AI总结：
本文通过定义基于对数概率差异的因果影响度量，并利用电路发现技术，研究视觉语言模型幻觉输出的反事实鲁棒性，推导出检测不稳定所需的最小反事实样本数。
摘要：
视觉语言模型（VLM）已知会产生不基于视觉证据的幻觉预测，但现有方法缺乏对这些预测在反事实扰动下鲁棒性的原则性理解。在这项工作中，我们研究了VLM中幻觉输出的反事实鲁棒性的样本复杂度。我们基于事实、反事实和激活修补运行之间的对数概率差异定义了一个因果影响度量，并用它来表征幻觉预测的稳定性。通过利用电路发现技术（CD-T），我们识别负责这些预测的模型组件，并追踪它们在反事实样本中的激活差异。然后，我们利用浓度不等式和因果影响分布的方差估计，推导出可靠检测幻觉输出不稳定性所需的最小反事实样本数m的经验界限。
英文摘要：
Visual Language Models (VLMs) are known to produce hallucinated predictions that are not grounded in visual evidence, yet existing approaches lack a principled understanding of how robust such predictions are under counterfactual perturbations. In this work, we study the sample complexity of counterfactual robustness for hallucinated outputs in VLMs. We define a causal influence metric based on log-probability differences between factual, counterfactual, and activation-patched runs, and use it to characterize the stability of hallucinated predictions. By leveraging circuit discovery techniques (CD-T), we identify model components responsible for these predictions and track their activation differences across counterfactual samples. We then derive empirical bounds on the minimum number of counterfactual samples m required to reliably detect instability in hallucinated outputs, using concentration inequalities and variance estimates of the causal influence distribution.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
