Reasoning as Gradient: Scaling MLE Agents Beyond Tree Search
Nanjing University(南京大学);Microsoft Research Asia(微软亚洲研究院);Peking University(北京大学);Zhejiang University(浙江大学);Wuhan University(武汉大学);Hong Kong University of Science and Technology(香港科技大学)
LLM-based agents for machine learning engineering (MLE) predominantly rely on tree search, a form of gradient-free optimization that uses scalar validation scores to rank candidates. As LLM reasoning capabilities improve, exhaustive enumeration becomes increasingly inefficient compared to directed updates, analogous to how accurate gradients enable efficient descent over random search. We introduce Gome, an MLE agent that operationalizes gradient-based optimization. Gome maps structured diagnostic reasoning to gradient computation, success memory to momentum, and multi-trace execution to distributed optimization. Under a closed-world protocol that isolates architectural effects from external knowledge, Gome achieves a state-of-the-art 35.1\% any-medal rate on MLE-Bench with a restricted 12-hour budget on a single V100 GPU. Scaling experiments across 10 models reveal a critical crossover: with weaker models, tree search retains advantages by compensating for unreliable reasoning through exhaustive exploration; as reasoning capability strengthens, gradient-based optimization progressively outperforms, with the gap widening at frontier-tier models. Given the rapid advancement of reasoning-oriented LLMs, this positions gradient-based optimization as an increasingly favorable paradigm. We release our codebase and GPT-5 traces at https://github.com/microsoft/RD-Agent.
Scheduling Your LLM Reinforcement Learning with Reasoning Trees
Cranberry-Lemon University(Cranberry-Lemon 大学);University of the Witwatersrand(独立研究者);Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州))
Interpretable Contrastive Monte Carlo Tree Search Reasoning
The University of Sydney(悉尼大学);Peking University(北京大学);Xiaohongshu Inc.(小红书公司);Shanghai AI Laboratory(上海人工智能实验室);Tsinghua University(清华大学);The Hong Kong University of Science and Technology(香港科技大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Search
Stanford University(斯坦福大学);University of Tokyo(东京大学);RIKEN AIP(日本理化学研究所革新智能研究中心);UC Berkeley(加州大学伯克利分校);University of Washington(华盛顿大学);Amazon AWS(亚马逊云科技)
Re-ranking Reasoning Context with Tree Search Makes Large Vision-Language Models Stronger
School of Artificial Intelligence, University of Chinese Academy of Sciences, China(中国科学院大学人工智能学院);MAIS, Institute of Automation, Chinese Academy of Sciences, China(中国科学院自动化研究所MAIS部);Alibaba Cloud Computing, China(阿里巴巴云计算)
Breaking the Reward Barrier: Accelerating Tree-of-Thought Reasoning via Speculative Exploration
Institute for Artificial Intelligence, Peking University School of Integrated Circuits, Peking University(人工智能研究院,北京大学集成电路学院,北京大学);School of Integrated Circuits, Peking University(集成电路学院,北京大学);ByteDance Seed(字节跳动种子)
Advancing LLM Reasoning Generalists with Preference Trees
Tsinghua University(清华大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Northeastern University(东北大学);Renmin University of China(中国人民大学);BUPT(北京邮电大学);Tencent(腾讯)
Adaptive Uncertainty-Aware Tree Search for Robust Reasoning
Institute of Software Chinese Academy of Sciences, Beijing, China(中国科学院软件研究所);University of the Chinese Academy of Sciences(中国科学院大学);Multimodal Experiences Lab, Dolby Laboratories Inc(多模态体验实验室,Dolby Laboratories Inc);Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University(人工智能与机器人研究所,西安交通大学);Electronic Information Science and Technology, Central South University(电子信息科学与技术,中南大学)
Advancing Tool-Augmented Large Language Models: Integrating Insights from Errors in Inference Trees
National Key Laboratory for Novel Software Technology(计算机软件新技术国家重点实验室);School of Artificial Intelligence, Nanjing University(南京大学人工智能学院);Alibaba International Digital Commerce(阿里巴巴国际数字商业集团);Pazhou Laboratory (Huangpu)(琶洲实验室(黄埔))
Guided Reasoning in LLM-Driven Penetration Testing Using Structured Attack Trees
Department of Electrical and Computer Engineering, Rochester Institute of Technology(罗切斯特理工学院电子与计算机工程系);Institute for Informatics and Applied Technology, Gonzaga University(贡萨加大学信息学与应用技术研究所)