arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.04809cs.AI
收藏

SCALER:合成可扩展自适应学习环境用于推理

SCALER:Synthetic Scalable Adaptive Learning Environment for Reasoning

Fudan University(复旦大学);Shanghai Innovation Institute(上海创新研究院);Singapore Management University(新加坡国立大学)

Caijun Xu, Changyi Xiao, Zhongyuan Peng, Xinrun Wang, Yixin Cao

AI总结:

SCALER通过自适应环境设计维持有效学习信号,解决RL训练中任务难度与模型能力不匹配及训练模式狭窄的问题,提升推理能力。

摘要:

SCALER通过自适应环境设计维持有效学习信号,解决RL训练中任务难度与模型能力不匹配及训练模式狭窄的问题,提升推理能力。

英文摘要:

Reinforcement learning (RL) offers a principled way to enhance the reasoning capabilities of large language models, yet its effectiveness hinges on training signals that remain informative as models evolve. In practice, RL progress often slows when task difficulty becomes poorly aligned with model capability, or when training is dominated by a narrow set of recurring problem patterns. To jointly address these issues, we propose SCALER (Synthetic sCalable Adaptive Learning Environment for Reasoning), a framework that sustains effective learning signals through adaptive environment design. SCALER introduces a scalable synthesis pipeline that converts real-world programming problems into verifiable reasoning environments with controllable difficulty and unbounded instance generation, enabling RL training beyond finite datasets while preserving strong correctness guarantees. Building on this, SCALER further employs an adaptive multi-environment RL strategy that dynamically adjusts instance difficulty and curates the active set of environments to track the model's capability frontier and maintain distributional diversity. This co-adaptation prevents reward sparsity, mitigates overfitting to narrow task patterns, and supports sustained improvement throughout training. Extensive experiments show that SCALER consistently outperforms dataset-based RL baselines across diverse reasoning benchmarks and exhibits more stable, long-horizon training dynamics.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑