arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2605.27885cs.CV

教师与求解器智能体之间的反思性对话用于视频问答

Reflective Dialogue between Teacher and Solver Agents for Video Question Answering

Nagoya Institute of Technology(名古屋技术大学)

Takuya Murakawa, Toru Tamaki

AI总结:

提出一种仅通过推理时上下文注入的适应方法,利用教师与求解器智能体之间的反思性对话(RD)来提升视频问答性能,在EgoCross基准上超越零样本和标准上下文学习,获得CVPR 2026 EgoVis Workshop跨域挑战赛开源赛道第三名。

摘要:

已经提出了各种方法来使视觉语言模型(VLM)适应视频问答的专门领域,包括微调和上下文学习。然而,在推理阶段仅从少量标记支持集获取任务特定知识而不进行微调仍然是一个挑战。在本文中,我们提出了一种仅通过推理时上下文注入来实现适应的方法。我们的方法首先构建一个反思性对话(RD)——两个智能体之间的多轮对话,其中教师提出每个支持问题并提供正确性反馈,求解器回答并提供视觉基础解释(或反思)以说明正确和错误的答案。然后,该对话历史在推理阶段用作上下文。在EgoCross基准上的实验表明,我们的方法优于基线零样本设置和直接传递支持集示例的标准上下文学习方法,在CVPR 2026 EgoVis Workshop的第一届跨域EgoCross挑战赛开源赛道中获得第三名,本文也作为该挑战赛的技术报告。

英文摘要:

Various approaches have been proposed to adapt Vision-Language Models (VLMs) to specialized domains for Video Question Answering, including fine-tuning and in-context learning. However, acquiring task-specific knowledge at the inference phase from only a small labeled support set without fine-tuning remains a challenge. In this paper, we propose a method that achieves adaptation solely through inference-time context injection. Our method first constructs a Reflective Dialogue (RD) -- a multi-turn conversation between two agents, in which Teacher poses each support question and delivers correctness feedback, and Solver answers and provides visual grounding explanations (or reflections) for both correct and incorrect answers. This dialogue history is then used as context at the inference phase. Experiments on the EgoCross benchmark demonstrate that our method outperforms both a baseline zero-shot setting and a standard in-context learning approach that passes support set examples directly, achieving 3rd place in the Open-source Track of the 1st Cross-Domain EgoCross Challenge at the CVPR 2026 EgoVis Workshop, for which this paper also serves as a technical report.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑