MMCL-Bench：多模态上下文学习从视觉规则、程序和证据
MMCL-Bench: Multimodal Context Learning from Visual Rules, Procedures, and Evidence
AI总结：
MMCL-Bench提出一个多模态上下文学习基准，要求模型从视觉或混合模态教学上下文中学习规则、程序和经验模式，并应用于新实例。评估发现当前系统在多模态上下文学习上仍不稳健。
摘要：
我们介绍了MMCL-Bench，一个多模态上下文学习基准：从视觉或混合模态的教学上下文学习任务特定的规则、程序和经验模式，并将其应用于新的视觉实例。与纯文本上下文学习或标准多模态问答不同，此设置要求模型在能够对学习的上下文进行推理之前，必须从图像、截图、手册、视频和帧序列中恢复并定位相关证据。MMCL-Bench包含102个任务，涵盖三个类别：规则系统应用、程序任务执行和经验发现与归纳。我们使用严格评分标准评估前沿多模态模型，并发现当前系统在多模态上下文学习上仍远未稳健，即使最强的模型在严格评估下也只解决不到三分之一的任务。诊断消融和错误分析显示，失败发生在上下文到答案的整个流程中，包括上下文锚定、视觉证据提取、上下文推理和响应构建。因此，MMCL-Bench突显多模态上下文学习是当前多模态模型的重要未解决能力瓶颈。
英文摘要：
We introduce MMCL-Bench, a benchmark for multimodal context learning: learning task-local rules, procedures, and empirical patterns from visual or mixed-modality teaching context and applying them to new visual instances. Unlike text-only context learning or standard multimodal question answering, this setting requires models to recover and localize relevant evidence from images, screenshots, manuals, videos, and frame sequences before they can reason over the learned context. MMCL-Bench contains 102 tasks spanning three categories: rule system application, procedural task execution, and empirical discovery and induction. We evaluate frontier multimodal models with strict rubric-based scoring and find that current systems remain far from robust multimodal context learning, with even the strongest model solving fewer than one-third of tasks under strict evaluation. Diagnostic ablations and error analysis show that failures arise throughout the context-to-answer pipeline, including context anchoring, visual evidence extraction, context reasoning, and response construction. MMCL-Bench thus highlights multimodal context learning as an important unsolved capability bottleneck for current multimodal models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
