ToG-Bench：面向任务的时空接地在第一人称视频中
ToG-Bench: Task-Oriented Spatio-Temporal Grounding in Egocentric Videos
AI总结：
本文提出ToG-Bench，首个面向任务的时空视频接地基准，通过任务导向的接地、显式-隐式双接地和一对一多接地特性，评估多对象和显式-隐式对象接地性能，揭示任务导向时空视频接地的挑战与性能差距。
摘要：
面向一般具身智能的核心能力在于从第一人称视角定位任务相关的对象，即时空视频接地（STVG）。尽管近年来取得了进展，现有STVG研究仍主要集中在对象中心和描述性指令上，忽略了对具身智能体完成目标导向交互至关重要的任务导向推理。为弥合这一差距，我们引入ToG-Bench，首个针对第一人称视频的任务导向时空视频接地基准。ToG-Bench具有三个关键特性：（1）任务导向接地，要求基于预期任务识别和定位对象，而非简单描述；（2）显式-隐式双接地，目标对象可以是显式提及或通过上下文推理隐式推断；（3）一对一多接地，单条指令可能对应多个参与任务执行的对象。基于ScanNet视频，ToG-Bench包含100个标注片段，2704条任务导向接地指令，通过半自动化流程结合基础模型注释和人工校正构建。此外，我们引入一组针对多对象和显式-隐式对象接地的任务级评估指标，并系统评估了七种最先进的多语言大模型。广泛实验揭示了任务导向STVG的内在挑战和显式-隐式及多对象接地之间的显著性能差距，突显了在具身场景中感知与交互之间的困难。数据和代码将在：https://github.com/qaxuDev/ToG-Bench发布。
英文摘要：
A core capability towards general embodied intelligence lies in localizing task-relevant objects from an egocentric perspective, formulated as Spatio-Temporal Video Grounding (STVG). Despite recent progress, existing STVG studies remain largely confined to object-centric and descriptive instructions, neglecting the task-oriented reasoning that is crucial for embodied agents to accomplish goal-directed interactions. To bridge this gap, we introduce \textbf{ToG-Bench}, the first task-oriented spatio-temporal video grounding benchmark for egocentric videos. ToG-Bench is characterized by three key features: (1) \textbf{Task-oriented Grounding}, which requires identifying and localizing objects based on intended tasks rather than straightforward descriptions; (2) \textbf{Explicit-Implicit Dual Grounding}, where target objects can be either explicitly mentioned or implicitly inferred by contextual reasoning; (3) \textbf{One-to-Many Grounding}, where a single instruction may correspond to multiple objects involved in task execution. Built upon videos sourced from ScanNet, ToG-Bench comprises 100 annotated clips with 2,704 task-oriented grounding instructions, constructed via a semi-automated pipeline that combines foundation model annotation and human refinement. In addition, we introduce a set of task-level evaluation metrics tailored for multi-object and explicit-implicit object grounding, and systematically benchmark seven state-of-the-art MLLMs. Extensive experiments reveal the intrinsic challenges of task-oriented STVG and substantial performance gaps across explicit-implicit and multi-object grounding, highlighting the difficulty of bridging perception and interaction in embodied scenarios. Data and code will be released at: \href{https://github.com/qaxuDev/ToG-Bench}{https://github.com/qaxuDev/ToG-Bench}..
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
