arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.17236cs.CLcs.AIcs.IR

大语言模型赋能的个性化Web智能体

Large Language Models Empowered Personalized Web Agents

National University of Singapore(新加坡国立大学);The Hong Kong Polytechnic University(香港理工大学);University of Science and Technology of China(中国科学技术大学);Eastern Institute of Technology(宁波东方理工大学)

Hongru Cai, Yongqi Li, Wenjie Wang, Fengbin Zhu, Xiaoyu Shen, Wenjie Li, Tat-Seng Chua

AI总结:

针对现有基于大语言模型的Web智能体忽视个性化数据的问题,研究构建了个性化Web智能体基准PersonalWAB,提出PUMA框架,通过记忆检索与模型对齐实现个性化任务执行,性能优于现有方法。

摘要:

Web智能体已成为基于用户指令自动完成Web任务的一个极具前景的方向,显著提升了用户体验。近年来,Web智能体已从传统智能体演进为基于大语言模型(LLMs)的Web智能体。尽管取得了一定成效,现有的基于LLM的Web智能体忽视了个性化数据(如用户画像和历史Web行为)在辅助理解用户个性化指令、执行定制化动作方面的重要性。为克服这一局限,我们首先明确了LLM赋能的个性化Web智能体任务,该类智能体整合个性化数据与用户指令,以实现指令理解和动作执行的个性化。针对缺乏综合评估基准的问题,我们构建了个性化Web智能体基准(PersonalWAB),涵盖用户指令、个性化用户数据、Web功能,以及覆盖三项个性化Web任务的两种评估范式。此外,我们提出了个性化用户记忆增强对齐(PUMA)框架,用于让LLM适配个性化Web智能体任务。PUMA利用带有任务特定检索策略的记忆库筛选相关的历史Web行为,再基于这些行为通过微调与直接偏好优化对齐LLM,以实现个性化动作执行。大量实验验证了在PersonalWAB上,PUMA的性能优于现有Web智能体。

英文摘要:

Web agents have emerged as a promising direction to automate Web task completion based on user instructions, significantly enhancing user experience. Recently, Web agents have evolved from traditional agents to Large Language Models (LLMs)-based Web agents. Despite their success, existing LLM-based Web agents overlook the importance of personalized data (e.g., user profiles and historical Web behaviors) in assisting the understanding of users' personalized instructions and executing customized actions. To overcome the limitation, we first formulate the task of LLM-empowered personalized Web agents, which integrate personalized data and user instructions to personalize instruction comprehension and action execution. To address the absence of a comprehensive evaluation benchmark, we construct a Personalized Web Agent Benchmark (PersonalWAB), featuring user instructions, personalized user data, Web functions, and two evaluation paradigms across three personalized Web tasks. Moreover, we propose a Personalized User Memory-enhanced Alignment (PUMA) framework to adapt LLMs to the personalized Web agent task. PUMA utilizes a memory bank with a task-specific retrieval strategy to filter relevant historical Web behaviors. Based on the behaviors, PUMA then aligns LLMs for personalized action execution through fine-tuning and direct preference optimization. Extensive experiments validate the superiority of PUMA over existing Web agents on PersonalWAB.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑