Reinforcement learning (RL) has become a standard paradigm for post-training and aligning Large Language Models (LLMs), yet recent evidence suggests it faces a persistent "capability ceiling": unlike classical RL systems that discover novel strategies, RL for LLMs often acts as a mere refiner of patterns already latent in pre-trained weights. In this work, we identify a fundamental structural bottleneck: while classical RL relies on compact, informative Markov states, current LLM post-training formulations are tethered to an ever-expanding history of actions.
We revisit a classical principle long central to RL yet absent from LLM post-training: explicit Markov states. Theoretically, we provide rigorous guarantees demonstrating that leveraging estimated Markov states can significantly reduce sample complexity. Empirically, we show that introducing Markov states consistently breaks the performance boundaries of standard RL post-training across a suite of complex logic puzzles. Our findings suggest that moving beyond "history-as-state" modeling in favor of structured Markovian representations is essential for unlocking open-ended discovery and genuinely new reasoning capabilities in Generative AI.
Behind Harmful Compliance: Behavioral and Mechanistic Divergence Across LLM Jailbreaks
Department of Computer Science(计算机科学系);Luddy School of Informatics, Computing, and Engineering(信息学、计算与工程学院);Indiana University Bloomington(印第安纳大学布卢明顿分校)
G-Zero: Self-Play for Open-Ended Generation from Zero Data
Washington University in St. Louis(华盛顿大学圣路易斯分校);University of Virginia(弗吉尼亚大学);University of Maryland(马里兰大学);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校)
Metis-SPECS: Decoupling Multimodal Learning via Self-distilled Preference-based Cold Start
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);MAIS, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所MAIS);Meituan(美团)
WMIP: A Working-Memory-Inspired Programmable Probe of Cumulative State Tracking in Large Language Models
Tohoku University(东北大学);Lappeenranta-Lahti University of Technology LUT(拉彭兰塔-拉赫蒂理工大学);Texas A&M University(德克萨斯农工大学);Worcester Polytechnic Institute(伍斯特理工学院)
KernelGenBench: Can LLMs and Agents Write Efficient Kernels Across Operator Sources and Hardware Platforms?
Beijing Normal University(北京师范大学);Beijing Jiaotong University(北京交通大学);Institute of Automation, CAS(中国科学院自动化研究所);Peking University(北京大学);BAAI(北京智源人工智能研究院)
Parallel-R1: Towards Parallel Thinking via Reinforcement Learning
Tencent AI Lab Seattle(腾讯人工智能实验室西雅图);University of Maryland, College Park(马里兰大学帕克分校);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);City University of Hong Kong(香港城市大学);Washington University in St. Louis(圣路易斯华盛顿大学)