Integrating Large Language Models (LLMs) with external tools via multi-agent systems offers a promising new paradigm for decomposing and solving complex problems. However, training these systems remains notoriously difficult due to the credit assignment challenge, as it is often unclear which specific functional agent is responsible for the success or failure of decision trajectories. Existing methods typically rely on sparse or globally broadcast rewards, failing to capture individual contributions and leading to inefficient reinforcement learning. To address these limitations, we introduce the Shapley-based Hierarchical Attribution for Reinforcement Policy (SHARP), a novel framework for optimizing multi-agent reinforcement learning via precise credit attribution. SHARP effectively stabilizes training by normalizing agent-specific advantages across trajectory groups, primarily through a decomposed reward mechanism comprising a global broadcast-accuracy reward, a Shapley-based marginal-credit reward for each agent, and a tool-process reward to improve execution efficiency. Extensive experiments across various real-world benchmarks demonstrate that SHARP significantly outperforms recent state-of-the-art baselines, achieving average match improvements of 23.66% and 14.05% over single-agent and multi-agent approaches, respectively.
Shapley Counterfactual Credits for Multi-Agent Reinforcement Learning
Zhejiang University(浙江大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Shenzhen Institute of Artificial Intelligence and Robotics for Society(深圳市人工智能与机器人研究院)
STAS: Spatial-Temporal Return Decomposition for Multi-agent Reinforcement Learning
Renmin University of China(中国人民大学);Peking University(北京大学);King’s College London(伦敦国王学院);National Key Laboratory of General Artificial Intelligence, BIGAI(通用人工智能全国重点实验室)
Shapley-Coop: Credit Assignment for Emergent Cooperation in Self-Interested LLM Agents
Antai College of Economics and Management, Shanghai Jiao Tong University, Shanghai, China(上海交通大学安泰经济管理学院);School of Computer Science and Technology, East China Normal University, Shanghai, China(华东师范大学计算机科学与技术学院);School of Computer Science and Technology, Tongji University, Shanghai, China(同济大学计算机科学与技术学院)