SmartAgent：面向网络世界中具身个性化智能体的用户思维链
SmartAgent: Chain-of-User-Thought for Embodied Personalized Agent in Cyber World
AI总结：
本文提出 COUT 用户思维链范式及 SmartAgent 框架，并构建 SmartSpot 数据集，使具身智能体能在网络 GUI 环境中推理用户显式与隐式个性化需求。
摘要：
近来，基于大型视觉语言模型（LVLM）、具备多模态感知与推理能力的具身智能体取得进展，擅长在真实或网络世界中自主交互，帮助人们在复杂环境中作出智能决策。然而，当前工作通常借助黄金动作轨迹或面向确定目标的理想任务解决方案进行优化。该范式考虑的面向用户因素有限，这可能是其在广泛个人助理应用中性能下降的原因。为解决该问题，我们提出用户思维链（COUT），这是一种新型具身推理范式，采用从基础动作思维到显式与隐式个性化偏好思维的思维链，将个性化因素纳入自主智能体学习。为实现 COUT，我们引入 SmartAgent，这是一个感知网络环境并推理个性化需求的智能体框架：1）与 GUI 交互以访问物品池，2）生成由先前动作隐含的用户显式需求，3）推荐物品以满足用户的隐式需求。为展示 SmartAgent 的能力，我们还创建了全新数据集 SmartSpot，提供全阶段、涉及个性化动作的环境。据我们所知，我们的工作首次形式化 COUT 过程，是迈向具身个性化智能体学习的初步尝试。我们在 SmartSpot 上的大量实验阐明了 SmartAgent 在一系列具身与个性化子任务中的功能。论文录用通知后，我们将在 https://github.com/tsinghua-fib-lab/SmartAgent 发布代码和数据。
英文摘要：
Recent advances in embodied agents with multimodal perception and reasoning capabilities based on large vision-language models (LVLMs), excel in autonomously interacting either real or cyber worlds, helping people make intelligent decisions in complex environments. However, the current works are normally optimized by golden action trajectories or ideal task-oriented solutions toward a definitive goal. This paradigm considers limited user-oriented factors, which could be the reason for their performance reduction in a wide range of personal assistant applications. To address this, we propose Chain-of-User-Thought (COUT), a novel embodied reasoning paradigm that takes a chain of thought from basic action thinking to explicit and implicit personalized preference thought to incorporate personalized factors into autonomous agent learning. To target COUT, we introduce SmartAgent, an agent framework perceiving cyber environments and reasoning personalized requirements as 1) interacting with GUI to access an item pool, 2) generating users' explicit requirements implied by previous actions, and 3) recommending items to fulfill users' implicit requirements. To demonstrate SmartAgent's capabilities, we also create a brand-new dataset SmartSpot that offers a full-stage personalized action-involved environment. To our best knowledge, our work is the first to formulate the COUT process, serving as a preliminary attempt towards embodied personalized agent learning. Our extensive experiments on SmartSpot illuminate SmartAgent's functionality among a series of embodied and personalized sub-tasks. We will release code and data upon paper notification at https://github.com/tsinghua-fib-lab/SmartAgent.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
