arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.24320cs.CL
收藏

DPEPO:基于大语言模型代理的多样化并行探索策略优化

DPEPO: Diverse Parallel Exploration Policy Optimization for LLM-based Agents

University of Electronic Science and Technology of China(电子科技大学);DiDi(滴滴)

Junshuo Zhang, Chengrui Huang, Feng Guo, Zihan Li, Ke Shi, Menghua Jiang, Jiguo Yu, Shuo Shang, Shen Gao

AI总结:

本文提出DPEPO算法,通过并行探索提升大语言模型代理在复杂任务中的性能,实验表明其在ALFWorld和ScienceWorld上达到最先进的成功率。

摘要:

基于顺序

英文摘要:

Large language model (LLM) agents that follow the sequential "reason-then-act" paradigm have achieved superior performance in many complex tasks.However, these methods suffer from limited exploration and incomplete environmental understanding, as they interact with only a single environment per step. In this paper, we first introduce a novel paradigm that enables an agent to interact with multiple environments simultaneously and share cross-trajectory experiences. Building upon this paradigm, we further propose DPEPO, a reinforcement learning (RL) algorithm that encourages the agent to perform diverse parallel exploration. There are two stages in DPEPO: initial supervised fine-tuning (SFT) imparts basic parallel reasoning and action generation, followed by reinforcement learning stage with a hierarchical reward scheme. We design a parallel trajectory-level success reward and two step-level rewards: Diverse Action Reward and Diverse State Transition Reward, which actively penalize behavioral redundancy and promote broad exploration. Extensive experiments on ALFWorld and ScienceWorld show that DPEPO achieves state-of-the-art (SOTA) success rates, while maintaining comparable efficiency to strong sequential baselines. (Code is available at https://github.com/LePanda026/Code-for-DPEPO)

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑