WARC-Bench：基于网络存档的GUI子任务执行基准
WARC-Bench: Web Archive Based Benchmark for GUI Subtask Executions
AI总结：
本文提出WARC-Bench，一个基于网络存档的GUI子任务执行基准，通过438个任务评估多模态AI代理在子任务上的能力，实验表明SFT和RLVR方法在提升子任务执行效果上取得显著成果。
摘要：
训练能够导航复杂现实网站的网络代理需要它们掌握子任务——多个UI组件上的短周期交互（例如在日期选择器中选择正确日期或在容器中滚动以提取信息）。我们介绍了WARC-Bench（网络存档基准），一个新型的网络导航基准，包含438个任务，旨在评估多模态AI代理在子任务上的能力。WARC-Bench利用Web ARChive文件实现动态且逼真的网页沙盒交互。我们证明WARC-Bench对领先的计算机使用模型具有挑战性，最高观察到的成功率仅为64.8%。为了提高开源模型在子任务上的表现，我们探索了两种常见的训练技术：监督微调（SFT）和具有可验证奖励的强化学习（RLVR）。实验表明，SFT模型在基准上的成功率为48.8%。在数据稀缺的情况下，通过RLVR训练SFT检查点，将分数提高到52.8%，在WARC-Bench上优于许多前沿模型。我们的分析得出结论：掌握这些子任务对于稳健的网络规划和导航至关重要，而这一能力并未被现有基准充分评估。
英文摘要：
Training web agents to navigate complex, real-world websites requires them to master $\textit{subtasks}$ - short-horizon interactions on multiple UI components (e.g., choosing the correct date in a date picker, or scrolling in a container to extract information). We introduce WARC-Bench (Web Archive Benchmark), a novel web navigation benchmark featuring 438 tasks designed to evaluate multimodal AI agents on subtasks. WARC-Bench enables sandboxed interactions with dynamic and realistic webpages using Web ARChive files. We show that WARC-Bench is challenging for leading computer-use models, with the highest observed success rate being 64.8%. To improve open source models on subtask, we explore two common training techniques: supervised fine-tuning (SFT) and reinforcement learning with verifiable rewards (RLVR). Experiments show that SFT models obtain a 48.8% success rate on the benchmark. Training with RLVR over SFT checkpoints, even in data-scarce settings, improves the score to 52.8% on WARC-Bench, outperforming many frontier models. Our analysis concludes that mastering these subtasks is essential for robust web planning and navigation, and is a capability not extensively evaluated by existing benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
