GuideMe：流视频中的多域任务指导与干预
GuideMe: Multi-Domain Task Guidance and Intervention in Streaming Video
AI总结：
构建流视频多域基准GuideMe，支持训练和评估用于闭环交互任务指导的MLLMs，设计评估框架衡量模型能力，实验发现现有MLLMs虽擅提供指令，但难识别执行错误并反馈。
摘要：
虽然多模态大语言模型（MLLMs）在离线视频理解方面表现出色，但它们距离成为实时程序教练还有多远仍是未知。本文构建了GuideMe，首个支持训练和评估用于闭环交互任务指导的MLLMs的流视频多域基准，设计评估框架并发现现有模型存在性能不对称问题。
英文摘要：
While multimodal Large Language Models (MLLMs) excel at offline video understanding, an interesting question of how far they are from serving as a real-time procedural coach remains unknown. Such a role typically requires an MLLM to continuously monitor the execution, detect mistakes, and provide corrective guidance in a closed-loop interaction. In this paper, we construct GuideMe, the first multi-domain benchmark for streaming video that supports training and evaluation of MLLMs for closed-loop interactive task guidance. It comprises 2,458 videos spanning 223.7 hours across diverse domains (\eg, cooking, object manipulation, daily-life guidance, and fitness), with 47,775 interaction samples covering next-step instructions, completion feedback, error detection, and corrective guidance. To evaluate existing models on GuideMe, we design a three-component assessment framework to measure the capabilities of representative MLLMs, which consists of temporal-semantic bipartite matching for sequence-level alignment, behavioral classification for intervention timing, and LLM-as-a-Judge for content quality. Extensive experiments highlight a critical performance asymmetry: despite excelling at providing instructions, existing MLLMs consistently fail to identify execution errors and respond with corrective feedback. Code and data are released at https://fawnliu.github.io/project/guideme.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
