Large Language Model (LLM) based agents have proved their ability to perform complex tasks like humans. However, there is still a large gap between open-sourced LLMs and commercial models like the GPT series. In this paper, we focus on improving the agent generalization capabilities of LLMs via instruction tuning. We first observe that the existing agent training corpus exhibits satisfactory results on held-in evaluation sets but fails to generalize to held-out sets. These agent-tuning works face severe formatting errors and are frequently stuck in the same mistake for a long while. We analyze that the poor generalization ability comes from overfitting to several manual agent environments and a lack of adaptation to new situations. They struggle with the wrong action steps and can not learn from the experience but just memorize existing observation-action relations. Inspired by the insight, we propose a novel AgentRefine framework for agent-tuning. The core idea is to enable the model to learn to correct its mistakes via observation in the trajectory. Specifically, we propose an agent synthesis framework to encompass a diverse array of environments and tasks and prompt a strong LLM to refine its error action according to the environment feedback. AgentRefine significantly outperforms state-of-the-art agent-tuning work in terms of generalization ability on diverse agent tasks. It also has better robustness facing perturbation and can generate diversified thought in inference. Our findings establish the correlation between agent generalization and self-refinement and provide a new paradigm for future research.
RAG-Star: Enhancing Deliberative Reasoning with Retrieval Augmented Verification and Refinement
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Wuhan University of Science and Technology（武汉科技大学）；National University of Singapore（新加坡国立大学）；Northeastern University at Qinhuangdao（东北大学秦皇岛分校）；BOSS Zhipin（BOSS直聘）
Enhancing the Medical Context-Awareness Ability of LLMs via Multifaceted Self-Refinement Learning
Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）；Huawei Noah’s Ark Lab（华为诺亚实验室）；College of AI, Tsinghua University（清华大学人工智能学院）；Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）
Enhancing LLM Steering through Sparse Autoencoder-Based Vector Refinement
LMU Munich（慕尼黑大学）；University of Georgia（佐治亚大学）；Northwestern University（西北大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；The Hong Kong Polytechnic University（香港理工大学）
Enhancing Ethical Explanations of Large Language Models through Iterative Symbolic Refinement
University of Manchester（曼彻斯特大学）；Idiap Research Institute（Idiap研究所）；National Biomarker Centre, CRUK-MI, University of Manchester（曼彻斯特大学英国癌症研究中心曼彻斯特研究所国家生物标志物中心）
Aligned but Fragile: Enhancing LLM Safety Robustness via Zeroth-Order Optimization
The State Key Laboratory of Blockchain and Data Security（区块链与数据安全国家重点实验室）；Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security（杭州高科技园区（滨江）区块链与数据安全研究院）；Sun Yat-sen University（中山大学）；KAUST（卡塔尔大学）
Enhancing LLM Reliability via Explicit Knowledge Boundary Modeling
X-LANCE Lab, School of Computer Science MoE Key Lab of Artificial Intelligence, SJTU AI Institute Shanghai Jiao Tong University, Shanghai, China（X-LANCE实验室，计算机科学学院，人工智能关键实验室，上海交通大学人工智能研究院，上海，中国）；Jiangsu Key Lab of Language Computing, Suzhou, China（语言计算重点实验室，苏州，中国）；AISpeech Co., Ltd., Suzhou, China（AISpeech公司，苏州，中国）；Center for Artificial Intelligence Research (CAiRE) Hong Kong University of Science and Technology, Hong Kong, China（人工智能研究中心（CAiRE）香港科技大学，香港，中国）；Suzhou Laboratory, Suzhou, China（苏州实验室，苏州，中国）
Retrieval is Not Enough: Enhancing RAG Reasoning through Test-Time Critique and Optimization
Zhejiang University（浙江大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；South China University of Technology（华南理工大学）；University of British Columbia（不列颠哥伦比亚大学）；Fudan University（复旦大学）；University of Hong Kong（香港大学）
MERLIN: Multimodal Embedding Refinement via LLM-based Iterative Navigation for Text-Video Retrieval-Rerank Pipeline
Buzzni AI Lab（Buzzni人工智能实验室）；KAIST（韩国科学技术院）；Mathpresso Inc.（Mathpresso公司）；UC Berkeley（加州大学伯克利分校）；Fainders AI（Fainders人工智能公司）；Seoul National University（首尔大学）
FRAME: Feedback-Refined Agent Methodology for Enhancing Medical Research Insights
South China University of Technology（南方科技大学）；HFIPS, Chinese Academy of Sciences（中国科学院HFIPS）；University of Science and Technology of China（中国科学技术大学）；The Third Affiliated Hospital of SuYat-sen University（孙逸德大学第三附属医院）
Training Language Model to Critique for Better Refinement
China Telecom Research Institute（中国电信研究院）；The Conversational Artificial Intelligence (CoAI) Group, Tsinghua University（清华大学对话人工智能（CoAI）小组）；University of Electronic Science and Technology of China（电子科技大学）；The Knowledge Engineering Group (KEG), Tsinghua University（清华大学知识工程小组）；Zhipu AI（智谱AI）
RPRO: Ranked Preference Reinforcement Optimization for Enhancing Medical QA and Diagnostic Reasoning
Department of Computer Science and Information Engineering, National Taiwan University of Science and Technology（计算机科学与信息工程系，台湾科技大学）；Department of Systems Engineering, Stevens Institute of Technology（系统工程系，史蒂文斯理工学院）；Department of Management Information Systems, National Chengchi University（管理信息系，National Chengchi University）；AI Research Center, Agaruda System（Agaruda系统人工智能研究中心）；Department of Computer Science, Stevens Institute of Technology（计算机科学系，史蒂文斯理工学院）；Far Eastern Memorial Hospital（东部纪念医院）；Smart Healthcare Interdisciplinary College, National Taipei University of Nursing and Health Sciences（智慧医疗跨领域学院，台北大学护理及健康科学学院）
COrAL: Order-Agnostic Language Modeling for Efficient Iterative Refinement
National University of Singapore（新加坡国立大学）；University of California, Santa Barbara（加州大学圣巴巴拉分校）；Mila（米拉研究所）；Nanyang Technological University（南洋理工大学）；University of Arizona（亚利桑那大学）
Hybrid Student-Teacher Large Language Model Refinement for Cancer Toxicity Symptom Extraction
Michigan State University（密歇根州立大学）；Henry Ford Cancer Institute（亨利·福特癌症研究所）；Cedars Sinai Medical Center（西达赛奈医疗中心）；New York University Abu Dhabi（纽约大学阿布扎比分校）
SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinement
University of California, Santa Barbara（加利福尼亚大学圣巴巴拉分校）；Moatless AI；Carnegie Mellon University（卡内基梅隆大学）；National University of Singapore（新加坡国立大学）；Mila（米拉研究所）
