BARISTA：一种多任务第一人称视角基准，用于组合视觉理解
BARISTA: A Multi-Task Egocentric Benchmark for Compositional Visual Understanding
AI总结：
BARISTA通过185个真实世界咖啡制作视频，提供密集标注的数据集，涵盖全自动、手冲和胶囊式流程，用于评估场景理解中的多任务能力，揭示任务家族间强变化及无主导模型家族。
摘要：
场景理解是通用物理智能的核心，视频是捕捉场景状态和时间动态的主要模态。然而，理解物理过程仍困难，因为模型需结合物体定位、手-物体交互、关系解析、时间推理和步骤级过程推理。现有基准通常单独评估这些能力，限制了对模型在过程任务中失败原因的诊断。我们引入BARISTA，一个密集标注的第一人称视角数据集和基准，包含185个真实世界咖啡制作视频，涵盖全自动、手冲和胶囊式流程。BARISTA提供经过验证的每帧场景图，将持久物体身份链接到掩码、跟踪、框、属性、类型关系、手-物体交互、活动和过程步骤。从这些图中，我们推导出零样本语言任务，涵盖短语定位、手-物体交互识别、指称、活动识别、关系提取和时间视觉问答。实验揭示任务家族间强变化及无主导模型家族，将BARISTA定位为过程视频理解的挑战性诊断基准。代码和数据集可在https://huggingface.co/datasets/ramblr/BARISTA获取。
英文摘要：
Scene understanding is central to general physical intelligence, and video is a primary modality for capturing both state and temporal dynamics of a scene. Yet understanding physical processes remains difficult, as models must combine object localization, hand-object interactions, relational parsing, temporal reasoning, and step-level procedural inference. Existing benchmarks usually evaluate these capabilities separately, limiting diagnosis of why models fail on procedural tasks. We introduce BARISTA, a densely annotated egocentric dataset and benchmark of 185 real-world coffee-preparation videos covering fully automatic, portafilter-based, and capsule-based workflows. BARISTA provides verified per-frame scene graphs linking persistent object identities to masks, tracks, boxes, attributes, typed relations, hand-object interactions, activities, and process steps. From these graphs, we derive zero-shot language-based tasks spanning phrase grounding, hand-object interaction recognition, referring, activity recognition, relation extraction, and temporal visual question answering. Experiments reveal strong variation across task families and no consistently dominant model family, positioning BARISTA as a challenging diagnostic benchmark for procedural video understanding. Code and dataset available at https://huggingface.co/datasets/ramblr/BARISTA.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
