何时教与教什么：面向Web智能体的预算感知在线自适应
When and What to Teach: Budget-Aware Online Adaptation for Web Agents
浏览论文内容
中文总结 AI 辅助
针对Web智能体在线部署中教学成本高的问题，提出预算感知框架，通过可解性门控和分数引导回合选择，在保持性能的同时大幅降低教师调用和训练计算。
中文摘要 AI 辅助
Web智能体在自动化复杂互联网任务方面取得了显著成功，但在真实环境中部署它们需要持续的在线自适应。鉴于部署强大的专有模型在商业上仍然成本高昂，从业者必须依赖轻量级本地模型，这些模型在部署后通过来自更强教师的在线教学进行演化。然而，标准的交互式反馈带来了高昂的成本。我们表明，传统的轨迹级偏好优化在无法解决的回合和冗余的执行步骤上都浪费了预算。为了解决这些低效问题，我们提出了\u201c带预算轨迹修剪的分数引导在线教学\u201d，这是一个预算感知框架，系统地编排\u201c何时\u201d和\u201c什么\u201d进行教学。具体来说，我们的框架集成了一个可解性感知的教师门控来决定\u201c何时\u201d查询教师模型，以及一个分数引导的回合选择机制来决定\u201c什么\u201d信息丰富的回合需要保留。在MiniWoB和TimeWarp上的大量实验表明，我们的方法在实现相当的首遍成功率的同时，平均减少了22.6%的教师调用和52.1%的学生训练计算量。我们的代码可在https://this https URL获取。
英文摘要
Web agents have achieved significant success in automating complex internet tasks but deploying them in real-world environments requires continuous online adaptation. Given that deploying powerful proprietary models remains commercially cost-prohibitive, practitioners must rely on lightweight local models that evolve post-deployment via online teaching from a stronger teacher. However, standard interactive feedback imposes prohibitive costs. We show that conventional trajectory-level preference optimization wastes budget on both unresolvable episodes and redundant execution turns. To resolve these inefficiencies, we propose \textbf{Score-Guided Online Teaching with Budgeted Trajectory Trimming}, a budget-aware framework that systematically orchestrates \textbf{when} and \textbf{what} to teach. Specifically, our framework integrates a solvability-aware teacher gate to dictate \textbf{when} to query the teacher model and a score-guided turn selection mechanism to decide \textbf{what} informative turns to retain. Extensive experiments on MiniWoB and TimeWarp demonstrate that our method achieves comparable first-pass success while reducing teacher calls by 22.6\% and student training compute by 52.1\% on average. Our code is available at https://github.com/zjw131f1fc/budgeted-online-teaching.
发表机构
- University of Electronic Science and Technology of China（电子科技大学）
- Institute of Software Chinese Academy of Sciences（中国科学院软件研究所）
机构由 AI 辅助整理，请以论文原文为准。
