LLM agents have emerged as powerful systems for tackling multi-turn tasks by interleaving internal reasoning and external tool interactions. Agentic Reinforcement Learning has recently drawn significant research attention as a critical post-training paradigm to further refine these capabilities. In this paper, we present AT$^2$PO (Agentic Turn-based Policy Optimization via Tree Search), a unified framework for multi-turn agentic RL that addresses three core challenges: limited exploration diversity, sparse credit assignment, and misaligned policy optimization. AT$^2$PO introduces a turn-level tree structure that jointly enables Entropy-Guided Tree Expansion for strategic exploration and Turn-wise Credit Assignment for fine-grained reward propagation from sparse outcomes. Complementing this, we propose Agentic Turn-based Policy Optimization, a turn-level learning objective that aligns policy updates with the natural decision granularity of agentic interactions. ATPO is orthogonal to tree search and can be readily integrated into any multi-turn RL pipeline. Experiments across seven benchmarks demonstrate consistent improvements over the state-of-the-art baseline by up to 1.84 percentage points in average, with ablation studies validating the effectiveness of each component. Our code is available at https://github.com/zzfoutofspace/ATPO.
TimeSage-MT: A Multi-Turn Benchmark for Evaluating Agentic Time Series Reasoning
University of Oxford（牛津大学）；VulpiVox Intelligence；Eindhoven University of Technology（埃因霍温理工大学）；Griffith University（格里菲斯大学）；Squirrel Ai Learning；East China Normal University（华东师范大学）
RTPO: Reverse-Turn Policy Optimization for Stabilizing Agentic RL Training
Adelaide University（阿德莱德大学）；The University of Sydney（悉尼大学）；Curtin University（科廷大学）；School of CSIT（计算机科学与信息技术学院）；ACFR（澳大利亚空间研究中心）；School of EECMS（电子工程、计算机与数学科学学院）
SSE-Bio: A Structured Self-Evolving Agent with Agentic Retrieval Policy for Multi-Hop Biomedical Reasoning
School of Computing Science, University of Glasgow（格拉斯哥大学计算科学学院）；Brigham and Women’s Hospital, Harvard Medical School, Harvard University（哈佛大学医学院附属布莱根妇女医院）；Language Technology Lab, University of Cambridge（剑桥大学语言技术实验室）；School of Natural and Computing Science, University of Aberdeen（阿伯丁大学自然与计算科学学院）；School of Cancer Sciences, University of Glasgow（格拉斯哥大学癌症科学学院）；Cancer Research UK Scotland Institute（英国癌症研究苏格兰研究所）
Agentic AI for Bilevel Long-Term Optimization of Policy-Driven Physical Layer Systems
Key Laboratory of EMW Information (MoE), College of Future Information Technology, Fudan University（复旦大学未来信息技术学院电磁波信息科学教育部重点实验室）；James Watt School of Engineering, University of Glasgow（格拉斯哥大学詹姆斯·瓦特工程学院）；School of Engineering, Edith Cowan University（埃迪斯科文大学工程学院）；Information Systems Technology and Design Pillar, Singapore University of Technology and Design（新加坡科技设计大学信息系统技术与设计系）
Entropy Pacing Policy Optimization for Multi-Task Agentic Reinforcement Learning
Generative AI Lab, College of Computing and Data Science, Nanyang Technological University（生成式人工智能实验室，南洋理工大学计算与数据科学学院）；Tongyi Lab, Alibaba Group（阿里集团通义实验室）
SEED: Self-Evolving On-Policy Distillation for Agentic Reinforcement Learning
Tsinghua University（清华大学）；Zhejiang University（浙江大学）；The Chinese University of Hong Kong（香港中文大学）；Nanyang Technological University（南洋理工大学）；Tongji University（同济大学）
SPO++: Stream-Aligned Policy Optimization for Asynchronous Agentic RL
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Duke University（杜克大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
