Reinforcement Learning (RL) has been shown to improve the capabilities of large language models (LLMs). However, applying RL to open-domain tasks faces two key challenges: (1) the inherent subjectivity of these tasks prevents the verifiable rewards as required by Reinforcement Learning with Verifiable Rewards (RLVR); (2) Reinforcement Learning from Human Feedback (RLHF) relies on external reward mechanisms. To overcome these limitations, we propose Self-Examining Reinforcement Learning (SERL), a novel self-improving framework where the LLM serves as both Actor and Judge. SERL introduces two synergistic reward mechanisms without any external signals. On the one hand, to improve the Actor's capability, we derive rewards from Copeland-style pairwise comparison judgments across a group of generated responses. On the other hand, a self-consistency reward that encourages coherent judgments is proposed to improve the Judge's reliability. This process refines the Judge's capability, which in turn provides a more robust reward for Actor. Experiments show that our method outperforms existing self-improvement training methods. SERL improves the LC win rate of Qwen3-8B on AlpacaEval 2 from 52.37% to 59.90%. To the best of our knowledge, our method achieves state-of-the-art performance among self-improving approaches. Furthermore, it achieves a performance comparable to significantly larger models like Qwen3-32B, demonstrating superior effectiveness and robustness on open-domain tasks.
Efficient Cross-Domain Offline Reinforcement Learning with Dynamics- and Value-Aligned Data Filtering
City University of Hong Kong;University of Illinois Urbana-Champaign;Tencent;Institute of Artificial Intelligence, China Telecom (TeleAI);Tsinghua University. Corresponding Author
A Comparison Between Decision Transformers and Traditional Offline Reinforcement Learning Algorithms
School of Mathematics, Statistics and Computer Science(数学、统计与计算机科学学院);University of KwaZulu-Natal(夸祖鲁-纳塔尔大学);Centre for Artificial Intelligence Research (CAIR)(人工智能研究中心)
CLARE: Conservative Model-Based Reward Learning for Offline Inverse Reinforcement Learning
Tsinghua University(清华大学);Tongji University(同济大学);University of California, Davis(加州大学戴维斯分校);Arizona State University(亚利桑那州立大学);Ohio State University(俄亥俄州立大学);Zhongguancun Laboratory(中关村实验室)
National Defense Academy of Japan(日本防卫学院);Research Center for Mathematical Engineering, Chiba Institute of Technology(数学工程研究中心,秋田工科大学);International Research Center for Neuro intelligence (WPI-IRCN), The University of Tokyo(神经智能国际研究中心(WPI-IRCN),东京大学)
Neural-to-Tree Policy Distillation with Policy Improvement Criterion
National Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室);Polixir Technologies(波尔克斯科技公司);NetEase Fuxi AI Lab(网易伏羲人工智能实验室);Zhejiang University(浙江大学)
Learning Communication Between Heterogeneous Agents in Multi-Agent Reinforcement Learning for Autonomous Cyber Defence
Department of Electrical and Computer Engineering, Royal Military College of Canada(加拿大皇家军事学院电子与计算机工程系);Defence Research and Development Canada(加拿大国防研究与发展加拿大);Department of Computer Engineering and Software Engineering, Polytechnique Montréal(蒙特利尔理工学院计算机工程与软件工程系)
Advancing Multimodal Reasoning: From Optimized Cold Start to Staged Reinforcement Learning
Zhejiang University(浙江大学);Fudan University(复旦大学);Soochow University(苏州大学);Shanghai AI Laboratory(上海人工智能实验室);The Chinese University of Hong Kong(香港中文大学)
Towards Efficient Online Exploration for Reinforcement Learning with Human Feedback
Department of Statistics and Data Science, The Chinese University of Hong Kong(统计与数据科学系,香港中文大学);Department of Statistics, University of Wisconsin-Madison(统计系,威斯康星大学麦迪逊分校)
Revolutionizing Genomics with Reinforcement Learning Techniques
University of Technology Sydney(悉尼科技大学);Chulalongkorn University(朱拉隆功大学);Deakin University(迪肯大学);Rutgers University(罗格斯大学);Universidad de Granada(格拉纳达大学);University of Leicester(莱斯特大学);Xi'an Jiaotong-Liverpool University(西交利物浦大学);University of New England(新英格兰大学);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);University of Zurich(苏黎世大学);UNSW SYDNEY(悉尼新南威尔士大学)
InfoBot: Transfer and Exploration via the Information Bottleneck
Mila(米拉研究所);University of Montreal(蒙特利尔大学);McGill University(麦吉尔大学);Deepmind(深度思维);Google Brain(谷歌大脑);University College London(伦敦大学学院);University of California, Berkeley(加州大学伯克利分校)
Multimodal Information Fusion for Chart Understanding: A Survey of MLLMs -- Evolution, Limitations, and Cognitive Enhancement
College of Computer Science, Sichuan University(四川大学计算机科学学院);Engineering Research Center of Machine Learning(机器学习工程研究中心);China Telecom Institute of AI(中国电信人工智能研究院)
The Conditions of Physical Embodiment Enable Generalization and Care
Institute for Advanced Consciousness Studies(先进意识研究所);Monash Centre for Consciousness and Contemplative Studies(莫纳什意识与冥想研究中心);University of Oxford(牛津大学);Topos Institute(拓斯研究所);Allen Discovery Center(艾伦发现中心);Okinawa Institute of Science and Technology(冲绳科学技术研究所);Max Planck Institute for Human Cognitive and Brain Sciences(马克斯·普朗克人类认知与脑科学研究所);University of Hertfordshire(赫特福德郡大学);Brain and Creativity Institute(大脑与创造力研究所)