Reinforcement Learning with Verifiable Rewards~(RLVR) has become a prominent paradigm to enhance the capabilities (i.e.\ long-context) of Large Language Models~(LLMs). However, it often relies on gold-standard answers or explicit evaluation rubrics provided by powerful teacher models or human experts, which are costly and time-consuming. In this work, we investigate unsupervised approaches to enhance the long-context capabilities of LLMs, eliminating the need for heavy human annotations or teacher models' supervision. Specifically, we first replace a few paragraphs with special placeholders in a long document. LLMs are trained through reinforcement learning to reconstruct the document by correctly identifying and sequencing missing paragraphs from a set of candidate options. This training paradigm enables the model to capture global narrative coherence, significantly boosting long-context performance. We validate the effectiveness of our method on two widely used benchmarks, RULER and LongBench~v2. While acquiring noticeable gains on RULER, it can also achieve a reasonable improvement on LongBench~v2 without any manually curated long-context QA data. Furthermore, we conduct extensive ablation studies to analyze the impact of reward design, data curation strategies, training schemes, and data scaling effects on model performance. We publicly release our code, data, and models.
Revisiting Entropy Regularization: Adaptive Coefficient Unlocks Its Potential for LLM Reinforcement Learning
State Key Lab of Processors, Institute of Computing Technology, CAS(处理器国家重点实验室,计算技术研究所,中国科学院);University of Science and Technology of China(中国科学技术大学);University of Chinese Academy of Sciences(中国科学院大学);StepFun Inc(StepFun公司)
INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);Beijing Academy of Artificial Intelligence(北京人工智能研究院);School of Computer Science, Beijing University of Posts and Telecommunications(北京邮电大学计算机学院);School of Artificial Intelligence, Nankai University(南开大学人工智能学院)
Instruction-Tuning Large Language Models with Medical Classification Data Improves ICD Coding of German Tumor Diagnoses
University Medical Center of the Johannes Gutenberg University Mainz(美因茨约翰内斯·古腾堡大学医学中心);University Cancer Center of the Johannes Gutenberg University Mainz(美因茨约翰内斯·古腾堡大学癌症中心)
BEAVER: A Training-Free Hierarchical Prompt Compression Method via Structure-Aware Page Selection
School of Computer Technology and Application, Qinghai University(青海大学计算机技术与应用学院);Tsinghua University(清华大学);Ant Group Security and Intelligence Laboratory (SIL)(蚂蚁集团安全与智能实验室)
Exploring Training and Inference Scaling Laws in Generative Retrieval
National University of Singapore(新加坡国立大学);The Hong Kong Polytechnic University(香港理工大学);University of Science and Technology of China(中国科学技术大学);Nanyang Technological University(南洋理工大学)
DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Search
Stanford University(斯坦福大学);University of Tokyo(东京大学);RIKEN AIP(日本理化学研究所革新智能研究中心);UC Berkeley(加州大学伯克利分校);University of Washington(华盛顿大学);Amazon AWS(亚马逊云科技)
Studying the Soupability of Documents in State Space Models
Department of Computer Science, University of California San Diego, San Diego, CA, USA(加州大学圣地亚哥分校计算机科学系);University of Illinois Urbana-Champaign, Champaign, IL, USA(伊利诺伊大学厄巴纳-香槟分校)
ContextRL: Enhancing MLLM's Knowledge Discovery Efficiency with Context-Augmented RL
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));Chinese Academy of Sciences(中国科学院);Tsinghua University(清华大学)