This paper presents PROB-IRM, an approach that learns robust reward machines (RMs) for reinforcement learning (RL) agents from noisy execution traces. The key aspect of RM-driven RL is the exploitation of a finite-state machine that decomposes the agent's task into different subtasks. PROB-IRM uses a state-of-the-art inductive logic programming framework robust to noisy examples to learn RMs from noisy traces using the Bayesian posterior degree of beliefs, thus ensuring robustness against inconsistencies. Pivotal for the results is the interleaving between RM learning and policy learning: a new RM is learned whenever the RL agent generates a trace that is believed not to be accepted by the current RM. To speed up the training of the RL agent, PROB-IRM employs a probabilistic formulation of reward shaping that uses the posterior Bayesian beliefs derived from the traces. Our experimental analysis shows that PROB-IRM can learn (potentially imperfect) RMs from noisy traces and exploit them to train an RL agent to solve its tasks successfully. Despite the complexity of learning the RM from noisy traces, agents trained with PROB-IRM perform comparably to agents provided with handcrafted RMs.
Noise-corrected GRPO: From Noisy Rewards to Unbiased Gradients
Department of Machine Learning, Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE；Technology Innovation Institute, Abu Dhabi, UAE；Department of Robotics, Khalifa University, Abu Dhabi, UAE
Reward Machines for Deep RL in Noisy and Uncertain Environments
University of Toronto（多伦多大学）；Vector Institute（矢量研究所）；Cornell University（康奈尔大学）；Pontificia Universidad Católica de Chile（智利天主教大学）；Centro Nacional de Inteligencia Artificial（国家人工智能中心）；Schwartz Reisman Institute for Technology and Society（施瓦茨·赖斯曼技术与社会研究所）
Noisy Symbolic Abstractions for Deep RL: A case study with Reward Machines
University of Toronto（多伦多大学）；Vector Institute（矢量研究所）；Schwartz Reisman Institute for Technology and Society（施瓦茨·赖斯曼技术与社会研究所）；Pontificia Universidad Católica de Chile（智利天主教大学）；Centro Nacional de Inteligencia Artificial（国家人工智能中心）
University of Toronto（多伦多大学）；Vector Institute（向量研究所）；University of California, Santa Cruz（加州大学圣克鲁兹分校）；University of Illinois, Urbana–Champaign（伊利诺伊大学厄巴纳-香槟分校）
Addressing Long-Tail Noisy Label Learning Problems: a Two-Stage Solution with Label Refurbishment Considering Label Rarity
College of Artificial Intelligence and Green Energy, National Yang Ming Chiao Tung University（国立阳明交通大学人工智慧与绿色能源学院）；University at Albany, SUNY（纽约州立大学奥尔巴尼分校）
Online Multi-Label Classification under Noisy and Changing Label Distribution
School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机与信息工程学院）；School of Computing, National University of Singapore（新加坡国立大学计算学院）
University of Michigan（密歇根大学）；Artifical Intelligence Innovation and Incubation Institute, Fudan University（复旦大学人工智能创新与产业研究院）；Michigan State University（密歇根州立大学）
Unleashing the Potential of Regularization Strategies in Learning with Noisy Labels
The University of Sydney（悉尼大学）；New York University（纽约大学）；Data61, CSIRO（澳大利亚联邦科学与工业研究组织Data61）；University of Science and Technology of China（中国科学技术大学）；Hong Kong Baptist University（香港浸会大学）
Selective-Supervised Contrastive Learning with Noisy Labels
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；The University of Sydney（悉尼大学）
CleanRM: Reward Modeling from Noisy Feedback with a Few Clean Feedback for RLHF
Zhejiang University（浙江大学）；Xiaohongshu Inc（小红书公司）；National University of Singapore（新加坡国立大学）；New York University（纽约大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Beihang University（北京航空航天大学）；Beijing Advanced Innovation Center for Big Data and Brain Computing（北京大数据与脑计算高精尖创新中心）；China’s Aviation System Engineering Research Institute（中国航空系统工程研究所）
BoMD: Bag of Multi-label Descriptors for Noisy Chest X-ray Classification
Australian Institute for Machine Learning, University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）；Harvard Medical School, Harvard University（哈佛医学院）；Centre for Vision, Speech and Signal Processing, University of Surrey（萨里大学视觉、语音与信号处理中心）
Calibrating Pre-trained Language Classifiers on LLM-generated Noisy Labels via Iterative Refinement
School of Computational Science \& Engineering Georgia Institute of Technology Atlanta GA USA；Scheller College of Business Georgia Institute of Technology Atlanta GA USA；Georgia Institute of Technology
