扩展扩展逻辑:代理元合成逻辑推理
Scaling the Scaling Logic: Agentic Meta-Synthesis of Logic Reasoning
AI总结:
本文提出SSLogic框架,通过代理迭代生成和优化可执行的生成器-验证器对,生成具有新规则和难度梯度的任务家族,提升训练效率。
摘要:
强化学习从可验证奖励(RLVR)受限于数据:现有合成管道依赖专家编写代码或固定模板,限制增长到实例级扰动。我们将可进化单元从问题实例转移到任务家族规范。SSLogic是一种代理元合成框架,其中LLM代理在闭合的生成-验证-优化循环中迭代编写和优化可执行的生成器-验证器对,生成具有新规则和难度梯度的任务家族,而不是旧规则的参数变化。多门验证协议——多策略共识加对抗性盲审,其中独立代理通过编写和执行代码解决每个实例——在进入训练前过滤不恰当的任务。从400个种子家族开始,两轮进化产生953个家族和21,389个可验证实例。三种收敛比较(步匹配、令牌匹配和大小控制在外部Enigmata数据上)一致显示进化数据的更高训练效用,Enigmata上SynLogic +5.2,AIME25 +3.0,BBH +5.5。细粒度KORBench评估揭示了逻辑(+13.2%)和操作(+9.6%)的改进,将结构进化与下游收益联系起来。代码:https://github.com/AdAstraAbyssoque/Scaling-the-Scaling-Logic
英文摘要:
Reinforcement Learning from Verifiable Rewards (RLVR) is bottlenecked by data: existing synthesis pipelines rely on expert-written code or fixed templates, confining growth to instance-level perturbations. We shift the evolvable unit from problem instances to task-family specifications. SSLogic is an agentic meta-synthesis framework in which LLM agents iteratively author and refine executable Generator-Validator pairs inside a closed Generate-Validate-Refine loop, producing families with new rules and difficulty gradients rather than parameter variations of old ones. A Multi-Gate Validation Protocol -- multi-strategy consensus plus Adversarial Blind Review, where independent agents solve each instance by writing and executing code -- filters ill-posed tasks before they enter training. Starting from 400 seed families, two evolution rounds yield 953 families and 21,389 verifiable instances. Three converging comparisons (step-matched, token-matched, and size-controlled on external Enigmata data) consistently show higher training utility of evolved data, with gains of SynLogic +5.2, AIME25 +3.0, and BBH +5.5 on Enigmata. Fine-grained KORBench evaluation reveals selective improvements in logic (+13.2%) and operation (+9.6%), linking structural evolution to downstream gains. Code: https://github.com/AdAstraAbyssoque/Scaling-the-Scaling-Logic
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。