arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.06960cs.CL
收藏

经验之树:低重复与隐式奖励环境下自演化智能体的结构化经验管理方案

FinEvolveBench: A Benchmark for Self-Evolving Agents on Low-Repetition Tasks with Implicit Rewards

Yining Zhu, Zihao Deng, Leiming Wang, Jingfei Lu, Junbo Wang, Yuke Li, Jikun Shen, Chuncheng Ran

AI总结:

针对低重复任务与隐式奖励环境,提出结构化经验管理方法ToE,通过组织、检索、验证和更新经验,在金融情绪预测基准上优于无经验基线。

摘要:

基于经验的自我演化对于LLM智能体至关重要,但现有基准通常假设明确的目标、稳定的任务模式和清晰的反馈。我们研究了一个更具挑战性的场景:具有隐式奖励的低重复任务,其中过去的经验难以重用,且反馈是延迟的、有噪声的且是结果层面的。我们引入了\textsc{FinEvolveBench},一个时间控制的金融情绪预测基准,将每日新闻驱动的预测与未来超额收益联系起来。我们进一步提出了经验之树(ToE),一种结构化的经验管理方法,用于组织、检索、验证和更新智能体的经验。实验表明,通用经验机制并不一致地优于无经验基线,而ToE实现了更强的整体性能。这些结果强调了在隐式奖励环境中,结构化经验管理对于自演化智能体的重要性。

英文摘要:

Large language model (LLM) agents increasingly rely on external experience to continually adapt to changing environments without modifying their underlying models. Recent experience mechanisms have demonstrated promising results across diverse tasks. However, their effectiveness is typically evaluated within individual benchmark settings, and how experience mechanisms generalize across different scenarios remains insufficiently explored. In this work, we present a scenario-oriented analysis of experience mechanisms for LLM agents. We characterize existing evaluation scenarios along four dimensions: outcome observability, credit assignment complexity, environmental dynamics, and experience reusability. Our analysis shows that existing benchmarks often evaluate experience mechanisms under scenarios where at least one dimension is comparatively favorable, leaving more challenging combinations of scenario properties underexplored. To address this gap, we introduce \textsc{FinEvolveBench}, a reproducible benchmark built on a chronological stream of rich financial news and market data that enables systematic evaluation of experience-based self-evolution under challenging experience regimes characterized by noisy feedback, ambiguous credit assignment, environmental non-stationarity, and limited experience reusability. Experiments show that existing approaches exhibit substantially reduced or inconsistent gains in this setting, highlighting the scenario-dependent nature of experience mechanisms and the challenge of maintaining valid experience under changing environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑