AI 中文总结
本文提出CalibForge系统，通过对抗性求解器校准合成5431个终端任务，训练的模型在多个基准测试中取得显著性能提升，验证了求解器相关可学习性的实用价值。
AI 中文摘要
训练终端智能体需要可执行且可验证的任务，这些任务不仅要可解决，还要对学习具有适当的挑战性。可执行验证确立了任务的可行性，但未揭示任务相对于给定求解器设置的表现。在本文中，我们提出了CalibForge，这是一种自主终端任务合成系统，它利用经过验证的求解器行为，通过对抗性求解器校准来修改候选任务。多求解器校准针对异构求解器池中的分歧，而对比求解器校准则针对指定的强通过/弱失败关系；两者都将求解器相关的可学习区域操作化为以已证明的可解决性为锚点。使用CalibForge，我们构建了5431个校准后的终端任务。我们的 ablation 实验表明，两种策略都比仅通过创作和验证或普通单求解器反馈产生更有效的监督。在完整任务集合上训练的模型在Terminal-Bench 2.0上达到32.58%，在SWE-bench Pro上达到47.57%。与相应的基础模型相比，在Terminal-Bench 2.0上的最大改进达到24.71个百分点，在SWE-bench Pro上达到27.68个百分点，在Doc2Repo上达到30.04个百分点。这些结果共同支持求解器相关的可学习性作为构建有效且可迁移的智能体训练数据的实用目标。
英文摘要
Training terminal agents requires executable and verifiable tasks that are not merely solvable, but appropriately challenging for learning. Executable validation establishes feasibility, yet does not reveal how a task behaves relative to a given solver setting. In this paper, we present CalibForge, an autonomous terminal-task synthesis system that uses verified solver behavior to revise candidate tasks through adversarial solver calibration. Multi-solver calibration targets disagreement within a heterogeneous solver pool, whereas contrastive solver calibration targets a designated strong-pass/weak-fail relation; both operationalize a solver-relative learnable zone anchored in demonstrated solvability. Using CalibForge, we construct 5,431 calibrated terminal tasks. Our ablations show that both strategies yield more effective supervision than authoring and validation alone or ordinary single-solver feedback. Models trained on the full collection achieve 32.58% and 47.57% on Terminal-Bench 2.0. The largest improvements over the corresponding base model reach 24.71 percentage points on Terminal-Bench 2.0, 27.68 points on SWE-bench Pro, and 30.04 points on Doc2Repo. Together, these results support solver-relative learnability as a practical target for constructing effective and transferable agent training data.
CommentsDataset: https://huggingface.co/datasets/AweAI-Team/CalibForge. Repository: https://github.com/AweAI-Team/CalibForge
