The success of many RL techniques heavily relies on human-engineered dense rewards, which typically demand substantial domain expertise and extensive trial and error. In our work, we propose DrS (Dense reward learning from Stages), a novel approach for learning reusable dense rewards for multi-stage tasks in a data-driven manner. By leveraging the stage structures of the task, DrS learns a high-quality dense reward from sparse rewards and demonstrations if given. The learned rewards can be \textit{reused} in unseen tasks, thus reducing the human effort for reward engineering. Extensive experiments on three physical robot manipulation task families with 1000+ task variants demonstrate that our learned rewards can be reused in unseen tasks, resulting in improved performance and sample efficiency of RL algorithms. The learned rewards even achieve comparable performance to human-engineered rewards on some tasks. See our project page (https://sites.google.com/view/iclr24drs) for more details.
Do It for HER: First-Order Temporal Logic Reward Specification in Reinforcement Learning (Extended Version)
DEIB, Politecnico di Milano(博洛尼亚工程学院,米兰理工大学);Politecnico di Milano(米兰理工大学);INESC-ID/Instituto Superior Técnico Universidade de Lisboa(里斯本大学超级学院/葡萄牙信息科学与技术研究所);Dipartimento di Informatica Università degli Studi di Milano(米兰大学信息学院)
Flow-Direct: Feedback-Efficient and Reusable Guidance for Flow Models via Non-Parametric Guidance Field
College of Computing and Data Science, Nanyang Technological University, Singapore(南洋理工大学计算与数据科学学院,新加坡);Centre for Frontier AI Research, Agency for Science, Technology and Research, Singapore(前沿人工智能研究中心,新加坡科技研究局);Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore(高性能计算研究所,新加坡科技研究局)
Online Bipartite Matching with Reusable Capacity under Non-Stationary Rewards
Leonard N. Stern School of Business, New York University(纽约大学莱昂纳德·N.斯特恩商学院);Georgia Institute of Technology(佐治亚理工学院);Qiuzhen College, Tsinghua University(清华大学求真书院);Tsinghua University(清华大学);Yau Mathematical Sciences Center(丘成桐数学科学中心);Beijing Institute of Mathematical Sciences and Applications(北京数学科学与应用研究院)
Multilingual Verifier Bias in RLVR: Benchmark, Rollout Diagnosis, and the Cross-Lingual Selection Bottleneck
School of Engineering, Institute of Science Tokyo(科学东京大学工程学院);College of Control Science and Engineering, Zhejiang University(浙江大学控制科学与工程学院);Department of Electrical and Computer Engineering, National University of Singapore(新加坡国立大学电气与计算机工程学院)
DeltaTTG: Training Multi-Turn Agents with a Token-to-Go Progress Meter
College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院);Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院)
Enhancing LLM Metacognition via Cognitive Pairwise Training
National Engineering Laboratory for Intelligent Information Processing, Academy of Mathematics and Physics, Chinese Academy of Sciences(智能信息处理国家工程实验室,中国科学院数学物理研究所);University of Science and Technology of China(中国科学技术大学)