Every Step Counts: Step-Level Credit Assignment for Tool-Integrated Text-to-SQL
Institute of Computer Science and Technology, Soochow University, China(苏州大学计算机科学与技术学院);Ant Digital Technologies, Ant Group(蚂蚁集团数字技术部);School of Management, University of Science and Technology of China(中国科学技术大学管理学院)
Tool-integrated Text-to-SQL parsing has emerged as a promising paradigm, framing SQL generation as a sequential decision-making process interleaved with tool execution. However, existing reinforcement learning approaches mainly rely on coarse-grained outcome supervision, resulting in a fundamental credit assignment problem: models receive the same reward for any trajectory that yields the correct answer, even when intermediate steps are redundant, inefficient, or erroneous. Consequently, models are encouraged to explore suboptimal reasoning spaces, limiting both efficiency and generalization. To address this problem, we propose FineStep, a novel framework for step-level credit assignment in tool-augmented Text-to-SQL. First, we introduce a reward design with independent process rewards to alleviate the signal sparsity of outcome supervision. Next, we present a step-level credit assignment mechanism to precisely quantify the value of each reasoning step. Finally, we develop a policy optimization method based on step-level advantages for efficient updates. Extensive experiments on BIRD benchmarks show that FineStep achieves state-of-the-art performance and reduces redundant tool interactions, with a 3.25% average EX gain over GRPO at the 4B scale.
CriticSearch: Fine-Grained Credit Assignment for Search Agents via a Retrospective Critic
Institute of Automation, CAS(中国科学院自动化研究所);School of Advanced Interdisciplinary Sciences, UCAS(清华大学交叉信息学院);School of Artificial Intelligence, UCAS(清华大学人工智能学院)
My FAULT: Self-Diagnosis as Credit Assignment in Self-Evolving Agentic Reinforcement Learning
Alibaba Group(阿里巴巴集团);Kyoto University(京都大学);NII LLMC(国立情报学研究所LLMC);Peking University(北京大学);University of California, Los Angeles(加州大学洛杉矶分校);Arizona State University(亚利桑那州立大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Tsinghua University(清华大学);University of the Chinese Academy of Sciences(中国科学院大学)
FARCA: Fact-Aligned Reliability-Aware Credit Assignment for Reinforcement Learning with Factual Supervision
School of Computer Science and Engineering, Nanjing University of Science and Technology(南京理工大学计算机科学与工程学院);School of Intelligence Science and Technology, Nanjing University(南京大学智能科学与技术学院)
Of Errors and Echoes: Responsibility-Aware Credit Assignment for Tool-Use Agents
University of California, Irvine(加利福尼亚大学欧文分校);Georgia Institute of Technology(佐治亚理工学院);Dartmouth College(达特茅斯学院);Northeastern University(东北大学);University of Notre Dame(圣母大学)
Dr.Credit: Rubric-Grounded Process Credit Assignment for Deep Research Agents
University of Chinese Academy of Sciences(中国科学院大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Alibaba Group(阿里巴巴集团);Peking University(北京大学)
APEX-Searcher: Refining Credit Assignment with Subgoaling for Agentic Retrieval-Augmented Generation
University of Chinese Academy of Sciences(中国科学院大学);MAIS, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所MAIS部);Wenge Technology Co., Ltd(Wenger科技有限公司)
Outcome-Grounded Advantage Reshaping for Fine-Grained Credit Assignment in Mathematical Reasoning
Fudan University(复旦大学);XingYun lab, HUJING Digital Media & Entertainment Group(星云实验室,HUJING数字媒体与娱乐集团);University of Science and Technology Beijing(北京科技大学);Chinese Academy of Sciences(中国科学院);Beijing University of Posts and Telecommunications(北京邮电大学)