RegulAR：面向增强现实中流程任务的基于图的错误识别与辅助
RegulAR: Graph-Grounded Error Recognition and Assistance for Procedural Tasks in AR
浏览论文内容
中文总结 AI 辅助
RegulAR是一款结合分层依赖图与MLLM的AR任务助手，可识别流程任务错误并提供恢复指导，在被试内研究中被证实比仅用MLLM的基线系统更能帮助用户理解任务结构与恢复任务。
中文摘要 AI 辅助
在流程任务中错误不可避免，但大多数增强现实（AR）指导系统侧重于分步指令的传递，而非帮助用户识别错误并从中恢复。本文提出RegulAR，一款用于流程任务错误识别与恢复的AR任务助手。RegulAR将任务指令建模为分层依赖图，并将该结构与多模态大语言模型（MLLM）结合，以解释执行过程中的第一人称视角观察结果。这使RegulAR能够跟踪任务进度、按错误类型识别偏差、评估偏差对后续步骤的影响，并通过原位平视显示器提供适当突出的干预措施，该显示器可可视化任务状态与恢复指导。通过明确流程结构，RegulAR不仅支持下一步指导，还能推理出错在哪里、为何重要以及用户如何重回正轨。在一项被试内研究（N=12）中，参与者报告称使用RegulAR比仅使用MLLM基线系统能更好地理解任务结构并获得更好的恢复支持。
英文摘要
Errors are inevitable in procedural tasks, yet most AR guidance systems focus on step-by-step instruction delivery rather than helping users recognize and recover from mistakes. We present RegulAR, an AR task assistant for procedural error recognition and recovery. RegulAR models task instructions as a hierarchical dependency graph and combines this structure with a Multimodal Large Language Model (MLLM) to interpret egocentric observations during execution. This enables RegulAR to track progress, identify deviations by error type, estimate their impact on later steps, and deliver appropriately salient interventions through an in-situ head-up display that visualizes task state and recovery guidance. By making procedural structure explicit, RegulAR supports not only next-step guidance, but also reasoning about what went wrong, why it matters, and how users can get back on track. In a within-subject study (N=12), participants reported better task-structure understanding and recovery support with RegulAR than the MLLM-only baseline.
