M2K-VDG：模型自适应的多模态知识锚点增强视频对话生成
M2K-VDG: Model-Adaptive Multimodal Knowledge Anchor Enhanced Video-grounded Dialogue Generation
AI总结：
本文提出M2K-VDG框架，通过模型自适应的多模态知识锚点增强和反事实效应检测锚点标记，以缓解视频对话生成中的幻觉问题，并在三个基准上验证了其优越性。
摘要：
视频对话生成（VDG）要求系统基于多模态知识生成流畅且准确的回答。然而，多模态知识利用的困难在实践中给VDG模型带来了严重的幻觉问题。尽管先前的工作以多种方式缓解幻觉，但它们几乎没有注意到多模态知识锚点答案标记的重要性。在本文中，我们通过困惑度揭示不同的VDG模型会经历不同程度的幻觉，并表现出多样的锚点标记。基于这一观察，我们提出了M2K-VDG，一个用于减少幻觉的模型自适应多模态知识锚点增强框架。此外，我们引入反事实效应以实现更准确的锚点标记检测。在三个流行基准上的实验结果表明，我们的方法优于最先进的方法，证明了其在减少幻觉方面的有效性。
英文摘要：
Video-grounded dialogue generation (VDG) requires the system to generate a fluent and accurate answer based on multimodal knowledge. However, the difficulty in multimodal knowledge utilization brings serious hallucinations to VDG models in practice. Although previous works mitigate the hallucination in a variety of ways, they hardly take notice of the importance of the multimodal knowledge anchor answer tokens. In this paper, we reveal via perplexity that different VDG models experience varying hallucinations and exhibit diverse anchor tokens. Based on this observation, we propose M2K-VDG, a model-adaptive multimodal knowledge anchor enhancement framework for hallucination reduction. Furthermore, we introduce the counterfactual effect for more accurate anchor token detection. The experimental results on three popular benchmarks exhibit the superiority of our approach over state-of-the-art methods, demonstrating its effectiveness in reducing hallucinations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
