arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.15328cs.LGcs.AI
收藏

锚定与收获课程:一种用于Craftax的完全无掩码非大语言模型课程

Anchor-and-Harvest Curriculum: A Fully Unmasked Non-LLM Curriculum for Craftax

Brown University(布朗大学);York University(约克大学);Sun Yat-sen University(中山大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))

Jiayu Zhu, Luyuan Li, Yuyuan Shi, Hanyi Dong

AI总结:

本研究提出无LLM推理的AHC课程,在Craftax-Full任务上实现优于原生PPO-GTrXL的回报,性能接近LLM生成课程。

摘要:

我们提出了锚定与收获课程(Anchor-and-Harvest Curriculum,AHC),这是一种针对Craftax-Full的、基于领域知识的规则式训练系统。AHC结合了成就导向任务、收获的初始状态进展、难度自适应、有界选择以及原始任务实践,不使用任何大语言模型(LLM)推理,且保留全部43个可用动作。在2005401600次PPO rollout转换时,三次主要的AHC运行获得原生任务回报为46.70±0.57(样本标准差),而采用相同学生架构与评估协议的原生PPO-GTrXL的回报为41.91±1.01。评估时每个模型在1024个固定世界上各进行10次随机rollout。另外一组三次AHC运行达到46.08±1.23。两个单组件 ablation 保留了相近的平均回报,而同时冻结难度并移除继承进展则得到45.14±2.79。全部67项成就的表现显示,大部分回报差异来自中级成就,而最深层的成就仍未解决。使用14B教师的Skill-Preflight诊断显示,先决条件与楼层访问约束如何限制实践。AHC与三个14B诊断配置的总回报接近已报告的80B DiCode均值45.88,且低于235B均值48.55。这些结果表明,无需LLM推理的显式课程可在Craftax-Full上达到与LLM生成课程相当的回报,且无需动作掩码。

英文摘要:

We introduce Anchor-and-Harvest Curriculum (AHC), a domain-informed, rule-based training system for Craftax-Full. AHC combines achievement-oriented tasks, harvested initial-state progress, difficulty adaptation, bounded selection, and original-task practice. It uses no LLM inference and keeps all 43 actions available. At 2,005,401,600 PPO rollout transitions, three primary AHC runs obtain native-task return $46.70\pm0.57$ (sample SD), compared with $41.91\pm1.01$ for fresh native PPO-GTrXL with the same student architecture and evaluation protocol. Evaluation uses ten stochastic rollouts on each of 1024 fixed worlds per model. An additional cohort of three AHC runs reaches $46.08\pm1.23$. Two single-component ablations retain similar mean returns, while jointly freezing difficulty and removing inherited progress gives $45.14\pm2.79$. All-67-achievement profiles show that most of the return difference comes from Intermediate achievements, while the deepest-floor achievements remain unsolved. A Skill-Preflight diagnosis with a 14B teacher shows how prerequisite and floor-access constraints restrict practice. AHC and three 14B diagnostic configurations reach aggregate returns near the reported 80B DiCode mean of 45.88 and below the 235B mean of 48.55. These results show that an explicit curriculum without LLM inference can reach returns comparable to those reported for LLM-generated curricula on Craftax-Full, without action masking.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑