WIST：基于网页的迭代自我对战树用于领域针对性推理改进
WIST: Web-Grounded Iterative Self-Play Tree for Domain-Targeted Reasoning Improvement
AI总结：
WIST通过基于网页的迭代自我对战树框架，无需预设领域语料，直接从开放网络学习，提升领域推理能力，优于纯内生自我进化和语料基础自我对战基线。
摘要：
最近强化学习与可验证奖励（RLVR）的进步为语言模型的自我改进提供了实用路径，但现有方法面临关键权衡：内生自我对战可能在迭代中漂移，而语料基础方法依赖于精心编纂的数据环境。我们提出了WIST，一种基于网页的迭代自我对战树框架，用于领域针对性推理改进，直接从开放网络学习，无需任何预设领域语料。WIST逐步扩展领域树进行探索，并检索和清理路径一致的网络语料以构建可控的训练环境。然后进行挑战者-求解者自我对战，使用可验证奖励，并将学习能力信号反馈以更新节点后验并指导后续探索通过自适应课程。在四个基础模型上，WIST一致优于基模型，通常优于纯内生自我进化和语料基础自我对战基线，总体提升达到+9.8（Qwen3-4B-Base）和+9.7（OctoThinker-8B）。WIST也是领域可调节的，使Qwen3-8B-Base在医学上提升+14.79，使Qwen3-4B-Base在PhyBench上提升+5.28。消融进一步确认了WIST关键组件对稳定开放网络学习的重要性。我们的代码可在https://github.com/lfy-123/WIST获取。
英文摘要：
Recent progress in reinforcement learning with verifiable rewards (RLVR) offers a practical path to self-improvement of language models, but existing methods face a key trade-off: endogenous self-play can drift over iterations, while corpus-grounded approaches rely on curated data environments. We present \textbf{WIST}, a \textbf{W}eb-grounded \textbf{I}terative \textbf{S}elf-play \textbf{T}ree framework for domain-targeted reasoning improvement that learns directly from the open web without requiring any pre-arranged domain corpus. WIST incrementally expands a domain tree for exploration, and retrieves and cleans path-consistent web corpus to construct a controllable training environment. It then performs Challenger--Solver self-play with verifiable rewards, and feeds learnability signals back to update node posteriors and guide subsequent exploration through an adaptive curriculum. Across four backbones, WIST consistently improves over the base models and typically outperforms both purely endogenous self-evolution and corpus-grounded self-play baselines, with the Overall gains reaching \textbf{+9.8} (\textit{Qwen3-4B-Base}) and \textbf{+9.7} (\textit{OctoThinker-8B}). WIST is also domain-steerable, improving \textit{Qwen3-8B-Base} by \textbf{+14.79} in medicine and \textit{Qwen3-4B-Base} by \textbf{+5.28} on PhyBench. Ablations further confirm the importance of WIST's key components for stable open-web learning. Our Code is available at https://github.com/lfy-123/WIST.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
