外部知识对话视频（OKCV）数据集——围绕视频开展对话
Outside Knowledge Conversational Video (OKCV) Dataset -- Dialoguing over Videos
AI总结：
本研究将外部知识视觉问答拓展至视频对话场景，构建了含2017个视频、5986条标注对话的OKCV数据集，评估了多个基线模型并指出未来挑战，数据集已公开。
摘要：
在外部知识视觉问答（OK-VQA）任务中，模型必须识别图像中的相关视觉信息，并结合外部知识以准确回答问题。将该任务拓展至基于视频的视觉 grounding 对话场景后，对话模型不仅需要随时间推移识别相关视觉细节，还要回答所需信息不一定存在于视觉内容中的问题，此外还必须结合整体对话上下文开展后续交互。为探索这一任务，我们构建了一个包含2017个视频的数据集，配套5986条人工标注对话，共计40954个交替对话轮次。这些对话的上下文基于特定视频片段实现视觉 grounding，但问题还需要用到视觉内容中不存在的外部知识，因此模型不仅要定位相关视频片段，还需利用外部知识完成对话交互。我们还在该数据集上评估了多个基线模型，并指出了该任务未来面临的挑战。数据集已在以下地址公开：https://github.com/c-patsch/OKCV。
英文摘要：
In outside knowledge visual question answering (OK-VQA), the model must identify relevant visual information within an image and incorporate external knowledge to accurately respond to a question. Extending this task to a visually grounded dialogue setting based on videos, a conversational model must both recognize pertinent visual details over time and answer questions where the required information is not necessarily present in the visual information. Moreover, the context of the overall conversation must be considered for the subsequent dialogue. To explore this task, we introduce a dataset comprised of $2,017$ videos with $5,986$ human-annotated dialogues consisting of $40,954$ interleaved dialogue turns. While the dialogue context is visually grounded in specific video segments, the questions further require external knowledge that is not visually present. Thus, the model not only has to identify relevant video parts but also leverage external knowledge to converse within the dialogue. We further provide several baselines evaluated on our dataset and show future challenges associated with this task. The dataset is made publicly available here: https://github.com/c-patsch/OKCV.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
