Large language models (LLMs) have recently transformed from text-based assistants to autonomous agents capable of planning, reasoning, and iteratively improving their actions. While numerical reward signals and verifiers can effectively rank candidate actions, they often provide limited contextual guidance. In contrast, natural language feedback better aligns with the generative capabilities of LLMs, providing richer and more actionable suggestions. However, parsing and implementing this feedback effectively can be challenging for LLM-based agents. In this work, we introduce Critique-Guided Improvement (CGI), a novel two-player framework, comprising an actor model that explores an environment and a critic model that generates detailed nature language feedback. By training the critic to produce fine-grained assessments and actionable revisions, and the actor to utilize these critiques, our approach promotes more robust exploration of alternative strategies while avoiding local optima. Experiments in three interactive environments show that CGI outperforms existing baselines by a substantial margin. Notably, even a small critic model surpasses GPT-4 in feedback quality. The resulting actor achieves state-of-the-art performance, demonstrating the power of explicit iterative guidance to enhance decision-making in LLM-based agents.
Retrieval is Not Enough: Enhancing RAG Reasoning through Test-Time Critique and Optimization
Zhejiang University（浙江大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；South China University of Technology（华南理工大学）；University of British Columbia（不列颠哥伦比亚大学）；Fudan University（复旦大学）；University of Hong Kong（香港大学）
A Structured Framework for Evaluating and Enhancing Interpretive Capabilities of Multimodal LLMs in Culturally Situated Tasks
DJCAD, University of Dundee, United Kingdom（邓迪大学DJCAD部门）；ARG-tech, SSEN, University of Dundee, United Kingdom（邓迪大学）；School of Computer Science, University of Birmingham, United Kingdom（伯明翰大学计算机科学学院）
Double-Checker: Enhancing Reasoning of Slow-Thinking LLMs via Self-Critical Fine-Tuning
The Hong Kong University of Science and Technology（香港科学与技术大学）；University of Electronic Science and Technology of China（电子科技大学）；Dalian University of Technology（大连理工大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；Zhejiang University（浙江大学）；University of Oxford（牛津大学）；NVIDIA（NVIDIA公司）；University of Surrey（萨里大学）
DeepCritic: Deliberate Critique with Large Language Models
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学首都人工智能学院）；School of Computer Science and Technology, Beijing Jiaotong University（北京交通大学计算机科学与技术学院）
Training Language Model to Critique for Better Refinement
China Telecom Research Institute（中国电信研究院）；The Conversational Artificial Intelligence (CoAI) Group, Tsinghua University（清华大学对话人工智能（CoAI）小组）；University of Electronic Science and Technology of China（电子科技大学）；The Knowledge Engineering Group (KEG), Tsinghua University（清华大学知识工程小组）；Zhipu AI（智谱AI）
Self-Evolving Critique Abilities in Large Language Models
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Research Institute of Big Data（深圳大数据研究院）；Shenzhen International Center for Industrial and Applied Mathematics（深圳国际工业与应用数学中心）；Qwen Team, Alibaba Inc.（阿里云通义团队）
LongDPO: Unlock Better Long-form Generation Abilities for LLMs via Critique-augmented Stepwise Information
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（多媒体信息处理国家重点实验室，计算机科学学院，北京大学）；Pattern Recongnition Center, WechatAI, Tencent Inc（图案识别中心，微信AI，腾讯公司）；Dept. of Comp. Sci. & Tech., Tsinghua University（计算机科学与技术系，清华大学）
Beyond Sparse Rewards: Enhancing Reinforcement Learning with Language Model Critique in Text Generation
School of Computer Science, McGill University（麦吉尔大学计算机科学学院）；Department of Computer Science, University of Toronto（多伦多大学计算机科学系）；Mila – Québec AI Institute（米拉-魁北克人工智能研究所）；Google Research（谷歌研究院）
VL-GenRM: Enhancing Vision-Language Verification via Vision Experts and Iterative Training
The Hong Kong University of Science and Technology（香港科学与技术大学）；Nanyang Technological University（南洋理工大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Critic-CoT: Boosting the reasoning abilities of large language model via Chain-of-thoughts Critic
Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所信息处理实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Xiaohongshu Inc（小红书公司）
CriticSearch: Fine-Grained Credit Assignment for Search Agents via a Retrospective Critic
Institute of Automation, CAS（中国科学院自动化研究所）；School of Advanced Interdisciplinary Sciences, UCAS（清华大学交叉信息学院）；School of Artificial Intelligence, UCAS（清华大学人工智能学院）
What's Wrong with Your Code Generated by Large Language Models? An Extensive Study
Fudan University（复旦大学）；Peking University（北京大学）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Shanghai Qiji Zhifeng Co., Ltd.（上海启智风科技有限公司）；Huazhong University of Science and Technology（华中科技大学）
RoadMapper: A Multi-Agent System for Roadmap Generation of Solving Complex Research Problems
Beijing University of Posts and Telecommunications（北京邮电大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；IDEA Research（IDEA研究院）；Hithink RoyalFlush Information Network Co., Ltd.（Hithink RoyalFlush信息网络有限公司）
SIGMA: Refining Large Language Model Reasoning via Sibling-Guided Monte Carlo Augmentation
Beihang University（北京航空航天大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Leicester（莱斯特大学）；Nanyang Technological University（南洋理工大学）
