WebGen-Agent：利用多层级反馈与步骤级强化学习增强交互式网站生成
WebGen-Agent: Enhancing Interactive Website Generation with Multi-Level Feedback and Step-Level Reinforcement Learning
AI总结：
本文提出 WebGen-Agent，通过 VLM 截图与 GUI-agent 多层级反馈、回溯择优及 Step-GRPO 步骤级强化训练，显著提升交互式网站代码生成质量。
摘要：
由大型语言模型（LLM）驱动的智能体系统已在仓库级代码生成任务上展现出令人瞩目的性能。然而，对于网站代码库生成等严重依赖视觉效果和用户交互反馈的任务，当前代码智能体仅依靠简单的代码执行来获得反馈和验证。这种方法无法捕捉所生成代码的实际质量。本文提出 WebGen-Agent，这是一种新颖的网站生成智能体，利用全面且多层级的视觉反馈来迭代生成并完善网站代码库。视觉语言模型（VLM）会针对网站截图和 GUI-agent 测试生成详细且富有表现力的文本描述与建议，同时给出量化其质量的分数。截图分数和 GUI-agent 分数进一步与回溯及择优机制相结合，以提升智能体的性能。利用 WebGen-Agent 工作流程中固有的准确视觉分数，我们进一步提出“带截图与 GUI-agent 反馈的 Step-GRPO”，以提升 LLM 充当 WebGen-Agent 推理引擎的能力。通过在 Step-GRPO 中将每一步的截图和 GUI-agent 分数作为奖励，我们提供了密集且可靠的过程监督信号，从而有效提升模型的网站生成能力。在 WebGen-Bench 数据集上，WebGen-Agent 将 Claude-3.5-Sonnet 的准确率从 26.4% 提升至 51.9%，并将其外观分数从 3.0 提升至 3.9，优于此前最先进的智能体系统。此外，我们的 Step-GRPO 训练方法将 Qwen2.5-Coder-7B-Instruct 的准确率从 38.9% 提升至 45.4%，并将外观分数从 3.4 提升至 3.7。
英文摘要：
Agent systems powered by large language models (LLMs) have demonstrated impressive performance on repository-level code-generation tasks. However, for tasks such as website codebase generation, which depend heavily on visual effects and user-interaction feedback, current code agents rely only on simple code execution for feedback and verification. This approach fails to capture the actual quality of the generated code. In this paper, we propose WebGen-Agent, a novel website-generation agent that leverages comprehensive and multi-level visual feedback to iteratively generate and refine the website codebase. Detailed and expressive text descriptions and suggestions regarding the screenshots and GUI-agent testing of the websites are generated by a visual language model (VLM), together with scores that quantify their quality. The screenshot and GUI-agent scores are further integrated with a backtracking and select-best mechanism, enhancing the performance of the agent. Utilizing the accurate visual scores inherent in the WebGen-Agent workflow, we further introduce \textit{Step-GRPO with Screenshot and GUI-agent Feedback} to improve the ability of LLMs to act as the reasoning engine of WebGen-Agent. By using the screenshot and GUI-agent scores at each step as the reward in Step-GRPO, we provide a dense and reliable process supervision signal, which effectively improves the model's website-generation ability. On the WebGen-Bench dataset, WebGen-Agent increases the accuracy of Claude-3.5-Sonnet from 26.4% to 51.9% and its appearance score from 3.0 to 3.9, outperforming the previous state-of-the-art agent system. Additionally, our Step-GRPO training approach increases the accuracy of Qwen2.5-Coder-7B-Instruct from 38.9% to 45.4% and raises the appearance score from 3.4 to 3.7.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
