推进全模态具身智能体：从孤立技能到日常物理自主
Advancing Omnimodal Embodied Agents from Isolated Skills to Everyday Physical Autonomy
AI总结：
提出OmniAct框架，通过分层异步架构统一规划、记忆和验证，实现机器人在非结构化环境中的持久自主，在40项真实任务中提升成功率并降低token消耗。
摘要：
在非结构化环境中构建持久具身智能体需要统一编排跨网络（API、物联网）和物理（操作、导航）领域的异构工具，并能够自主从长时间运行中不可避免的物理故障中恢复。现有系统将这些视为分离的问题：基于VLM的规划器缺乏统一的网络-物理动作空间，智能体框架积累无界上下文导致时间连贯性下降，VLA策略执行开环而无法检测自身故障。我们认为持久自主性不需要单一模型，而需要一种分层异步架构，明确分离规划、记忆和验证。为此，我们提出OmniAct，一个集成多模态语义规划器以跨统一动作空间进行技能路由、具有基于事件边界压缩的自适应分层记忆以实现亚线性上下文增长，以及在物理执行期间关闭语义循环的异步视觉抢占引擎的框架。在两个协调四个物联网设备的机器人平台上进行的40个真实世界长时任务中，OmniAct在所有复杂度级别上实现了端到端成功率的持续提升，在超过10万累积交互token下保持近乎平坦的token消耗，并将中等规模开源模型提升至专有模型性能水平。
英文摘要：
Building persistent embodied agents in unstructured environments demands unified orchestration of heterogeneous tools spanning both cyber (APIs, IoT) and physical (manipulation, navigation) domains, coupled with autonomous recovery from physical failures that inevitably arise over extended operation. Existing systems treat these as separate problems: VLM-based planners lack a unified cyber-physical action space, agent frameworks accumulate unbounded context that degrades temporal coherence, and VLA policies execute open-loop without detecting their own failures. We argue that persistent autonomy requires not a monolithic model but a hierarchical asynchronous architecture with explicit separation of planning, memory, and verification. To this end, we present OmniAct, a framework integrating a multimodal semantic planner for skill routing across unified action spaces, an adaptive hierarchical memory with event-boundary-driven compression for sub-linear context growth, and an asynchronous visual preemption engine that closes the semantic loop during physical execution. Across 40 real-world long-horizon tasks on two robotic platforms coordinating four IoT devices, OmniAct achieves consistent improvements in end-to-end success across all complexity levels, maintains near-flat token consumption over under 100k+ accumulated interaction tokens, and elevates mid-scale open-weight models to proprietary-level performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
