发表机构
University of California, San Diego; Aether AI Lab(加利福尼亚大学圣地亚哥分校; 以太人工智能实验室)
机构由 AI 辅助整理,请以论文原文为准。AI 中文总结
研究针对长时程任务中智能体任务进展难解释、验证和恢复的问题,提出StructAgent框架,通过统一因果结构维护任务进展并规范工作流程,实验证明其能提升多种模型性能且具有通用性。
AI 中文摘要
大语言模型和视觉语言模型的进展使数字智能体能力增强,但现实世界任务常是长时程且情境不断演变,现有智能体基于原始交互历史操作,难以解释、验证和恢复任务进展。本文提出StructAgent,一个以状态为中心的框架,引入统一状态维护紧凑、可验证的任务进展,通过基于验证器的状态转移调节进展。在此基础上,StructAgent还实现了显式进度检查点、证据驱动任务完成、有针对性的故障恢复和工具支持的执行等功能。大量实验表明,StructAgent在长时程计算机使用任务上持续提升多种语言模型和视觉语言模型主干的性能,且该框架可推广到Minecraft等环境。
英文摘要
Recent advances in large language models (LLMs) and vision-language models (VLMs) have enabled increasingly capable digital agents for computer use. However, real-world tasks are often long-horizon and involve evolving contexts containing accumulated observations, intermediate edits, failed attempts, and partially completed executions. Existing agents typically operate over raw interaction history, making task progress difficult to interpret, verify, and recover, which ultimately limits reliable long-horizon execution. In this paper, we argue that addressing this challenge requires explicitly structuring both the agent's state and workflow around a unified causal representation of task progress. We present \textbf{StructAgent}, a state-centered framework that introduces a unified state for maintaining compact, verifiable task progress and a structured workflow that regulates progress through verifier-backed state transitions. Building on this design, StructAgent further enables explicit progress checkpointing, evidence-driven task completion, targeted failure recovery, and tool-supported execution, while ensuring that all progress updates remain grounded in verification. Extensive experiments demonstrate that StructAgent consistently improves a wide range of LLM and VLM backbones on long-horizon computer-use tasks. On OSWorld-Verified, it improves Qwen3.5-9B from 27.0\% to 46.9\% success rate and Qwen3.5-27B from 31.6\% to 62.2\%, while achieving a new open-source state of the art of 78.9\% with MiniMax-M3. Moreover, the same framework generalizes beyond desktop environments to Minecraft, demonstrating the generality of our design.