Trial and Error: Exploration-Based Trajectory Optimization for LLM Agents
Allen Institute for AI(艾伦人工智能研究所);School of Computer Science, Peking University(北京大学计算机学院);UCLA(加利福尼亚大学洛杉矶分校);Ohio State University(俄亥俄州立大学);UIUC(伊利诺伊大学厄巴纳-香槟分校)
Yifan Song, Da Yin, Xiang Yue, Jie Huang, Sujian Li, Bill Yuchen Lin
备注:
Accepted to ACL 2024 Main Conference; Camera Ready
Large Language Models (LLMs) have become integral components in various autonomous agent systems. In this study, we present an exploration-based trajectory optimization approach, referred to as ETO. This learning method is designed to enhance the performance of open LLM agents. Contrary to previous studies that exclusively train on successful expert trajectories, our method allows agents to learn from their exploration failures. This leads to improved performance through an iterative optimization framework. During the exploration phase, the agent interacts with the environment while completing given tasks, gathering failure trajectories to create contrastive trajectory pairs. In the subsequent training phase, the agent utilizes these trajectory preference pairs to update its policy using contrastive learning methods like DPO. This iterative cycle of exploration and training fosters continued improvement in the agents. Our experiments on three complex tasks demonstrate that ETO consistently surpasses baseline performance by a large margin. Furthermore, an examination of task-solving efficiency and potential in scenarios lacking expert trajectory underscores the effectiveness of our approach.
Carnegie Mellon University(卡内基梅隆大学);Massachusetts Institute of Technology(麻省理工学院);MIT-IBM Watson AI Lab(麻省理工学院-IBM沃森人工智能实验室);University of Michigan(密歇根大学);IBM Research(IBM研究院)
From Exploration to Mastery: Enabling LLMs to Master Tools via Self-Driven Interactions
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);Baidu Inc.(百度公司);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所)
When Experience Becomes Instruction: Trajectory Poisoning in Self-Evolving Agent Skill Systems
Ant Group(蚂蚁集团);Zhejiang University(浙江大学);Tsinghua University(清华大学);Hangzhou Dianzi University(杭州电子科技大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Xi’an Jiaotong University(西安交通大学)
Traj-LLM: A New Exploration for Empowering Trajectory Prediction with Pre-trained Large Language Models
Beihang University(北京航空航天大学);Research Institute for Frontier Science, Beihang University(北京航空航天大学前沿科学研究院);Beijing University of Technology(北京工业大学);College of Metropolitan Transportation, Beijing University of Technology(北京工业大学城市交通学院)
Teaching LLMs to Learn Tool Trialing and Execution through Environment Interaction
School of Computer Science and Engineering, Northeastern University(东北大学计算机科学与工程学院);Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系);Alibaba Group(阿里巴巴集团);School of Artificial Intelligence, Shanghai Jiao Tong University(上海交通大学人工智能学院)