正确的推理策略即是一切:面向EgoCross挑战的近乎无需训练的领域感知推理
The Right Inference Strategy Is All You Need: Nearly Training-Free Domain-Wise Inference for EgoCross Challenge
AI总结:
针对EgoCross挑战中源受限场景下多模态大模型在领域偏移严重的自我中心视频问答任务上表现不佳的问题,提出一种领域感知推理策略,通过为四个目标领域分别设计不同的输入、提示和答案映射流程,在不进行额外训练的情况下显著提升基线模型性能。
摘要:
EgoCross评估多模态大语言模型在显著领域偏移下的自我中心视频问答,其中测试视频来自手术、工业装配、极限运动和动物佩戴相机,而非日常场景。在源受限赛道中,基础模型固定为Qwen3-VL-4B,而官方任务特定支持集仅包含20个训练样本。这一设置使得挑战更侧重于向受限模型暴露正确的视觉、时序和答案选择线索,而非模型规模。我们的关键观察是,冻结的基线模型并非完全无法处理这些罕见场景;相反,它往往缺乏合适的接口来将其现有的视觉-语言知识迁移到新任务格式。因此,我们采用领域感知推理策略,将四个目标领域分开处理,并根据每个领域的任务特点设计不同的输入、提示和答案映射流程。这些策略通过强调每个领域重要的线索,使罕见自我中心场景对VLM更具可解释性。最终系统几乎无需训练:手术和动物问题使用基础Qwen3-VL-4B模型回答,而极限运动和工业问题仅使用在提供的20个训练样本上训练两个epoch的官方SFT检查点。在最终评估中,这一简单策略达到了66.98%的整体准确率,表明精心设计的领域感知推理可以弥补基础模型能力的不足,并恢复基线模型中已存在的大部分能力。
英文摘要:
EgoCross evaluates multimodal large language models on egocentric video question answering under substantial domain shift, where test videos come from surgery, industrial assembly, extreme sports, and animal-mounted cameras rather than ordinary daily-life scenes. In the source-limited track, the base model is fixed to Qwen3-VL-4B, while the official task-specific support set contains only 20 training samples. This setting makes the challenge less about model scaling and more about exposing the right visual, temporal, and answer-selection cues to a constrained model. Our key observation is that the frozen baseline model is not simply incapable of these rare scenarios; rather, it often fails to transfer its existing visual-language knowledge to the new task format without an appropriate interface. We therefore use a domain-wise inference strategy that treats the four target domains separately and designs different input, prompting, and answer-mapping procedures according to each domain's task characteristics. These strategies make the rare egocentric scenes more interpretable to the VLM by emphasizing the cues that matter for each domain. The resulting system is nearly training-free: surgery, and animal questions are answered with the base Qwen3-VL-4B model, while XSports and industry use only the official SFT checkpoint trained for two epochs on the provided 20 training samples. On the final evaluation, this simple strategy reaches 66.98\% overall accuracy, suggesting that careful domain-aware inference can compensate for limited base-model strength and recover much of the ability already present in the baseline model.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。