TIGER: 基于图证据路由的可追踪推理用于减轻多模态生成中的幻觉
Self-Correction Can Amplify Hallucinations: Fact-Level Repair with Graph-Based Evidence Routing in Multimodal Generation
AI总结：
提出TIGER框架，通过从输入和输出中独立提取观测图与声明图，并基于图条件风险评分修复高风险声明，以减轻多模态生成中的事实级幻觉。
摘要：
我们研究多模态生成的事实级修复，其中流畅的输出可能包含输入不支持的具体事实。现有的推理时修复方法通常通过联合条件化输入和当前输出来生成反馈。这种设计有两个局限性：输出中的幻觉声明可能偏置模型对输入的解释，且自由形式的反馈无法在事实级别进行排序或调度。我们提出TIGER，一种重新设计反馈以进行局部修复的推理时框架。TIGER从输入中独立提取观测图，从当前输出中提取声明图，然后根据支持和冲突为每个声明分配图条件风险分数。模型修复选定的高风险声明，同时保持骨干网络冻结。我们提供收敛性分析，表明在温和假设下，期望总风险几何级数下降至显式渐近界。跨四个跨模态路径（包括图像到文本、图像+文本到文本、音频到文本和视频到文本）的实验表明，TIGER在保持任务质量的同时减少了不支持内容。该增益在多个骨干网络上成立，CrisisFACTS案例研究表明相同的修复机制可以改善多源设置中的接地性。
英文摘要：
We study fact-level repair for multimodal generation, where a fluent output may contain specific facts that are not supported by the input. Existing inference-time repair methods often generate feedback by jointly conditioning on the input and the current output. This design has two limitations: hallucinated claims in the output can bias the model's interpretation of the input, and free-form feedback cannot be ranked or scheduled at the fact level. We present TIGER, an inference-time framework that redesigns feedback for localized repair. TIGER independently extracts an observation graph from the input and a claim graph from the current output, then assigns each claim a graph-conditioned risk score based on support and conflict. The model repairs selected high-risk claims while keeping the backbone frozen. We provide a convergence analysis showing that the expected total risk decreases geometrically to an explicit asymptotic bound under mild assumptions. Experiments across four cross-modal paths, including image-to-text, image+text-to-text, audio-to-text, and video-to-text, show that TIGER reduces unsupported content while preserving task quality. The gains hold across multiple backbones, and a CrisisFACTS case study suggests that the same repair mechanism can improve grounding in multi-source settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
