揭示隐藏联系：面向视频 grounding 对话的迭代搜索与推理
Uncovering Hidden Connections: Iterative Search and Reasoning for Video-grounded Dialog
AI总结：
本文提出一种迭代搜索与推理框架，通过文本路径搜索聚合对话核心线索、视觉迭代推理提取关键标记，并结合GPT-2生成视频对话答案，在三个数据集上验证了其有效性与泛化性。
摘要：
与传统的视觉问答不同，视频 grounding 对话要求对对话历史和视频内容都有深刻理解，才能生成准确的回复。尽管现有方法已取得可观进展，但它们仍面临逐步理解复杂对话历史和融合视频信息的挑战。为应对这些挑战，我们提出了一个迭代搜索与推理框架，该框架由文本编码器、视觉编码器和生成器组成。具体而言，我们在文本编码器中设计了一种路径搜索与聚合策略，从对话历史中挖掘对理解所提问题至关重要的核心线索。同时，我们的视觉编码器利用迭代推理网络提取并强调视频中的关键视觉标记，从而增强视觉理解的深度。最后，我们使用预训练的 GPT-2 模型作为答案生成器，将挖掘出的隐藏线索解码为连贯且符合语境的答案。在三个公开数据集上的大量实验证明了我们所提框架的有效性和泛化能力。
英文摘要：
In contrast to conventional visual question answering, video-grounded dialog necessitates a profound understanding of both dialog history and video content for accurate response generation. Despite commendable progress made by existing approaches, they still face the challenges of incrementally understanding complex dialog history and assimilating video information. In response to these challenges, we present an iterative search and reasoning framework, which consists of a textual encoder, a visual encoder, and a generator. Specifically, we devise a path search and aggregation strategy in the textual encoder, mining core cues from dialog history that are pivotal to understanding the posed questions. Concurrently, our visual encoder harnesses an iterative reasoning network to extract and emphasize critical visual markers from videos, enhancing the depth of visual comprehension. Finally, we utilize the pre-trained GPT-2 model as our answer generator to decode the mined hidden clues into coherent and contextualized answers. Extensive experiments on three public datasets demonstrate the effectiveness and generalizability of our proposed framework.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
