arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.03220cs.CLcs.AI
收藏

WebRISE: 面向MLLM生成Web工件的需求诱导状态评估

WebRISE: Requirement-Induced State Evaluation for MLLM-Generated Web Artifacts

Tsinghua University(清华大学);Huawei Noah’s Ark Lab(华为诺亚实验室);East China Normal University(华东师范大学);Tongji University(同济大学);Institute of Artificial Intelligence, Beihang University(北京航空航天大学人工智能研究院)

Yuxin Meng, Yuhan Suo, Junjie Wang, Yuhan Sun, Yiyao Yu, Ruixu Zhang, Ruining Hu, Yubin Wang, Shouwei Ruan, Bin Wang, Yuxiang Zhang, Yujiu Yang

AI总结:

提出WebRISE框架,通过交互契约图(ICG)将任务需求转化为可观察状态、用户意图转换和DOM/视觉断言,以评估MLLM生成的Web工件的功能正确性,实验表明ICG评分检测状态错误率是检查点评估的2-16倍。

摘要:

现有的MLLM生成Web工件基准通过局部证据评估交互,忽略了决定页面是否正常工作的需求诱导状态和转换。我们提出WebRISE,它将任务需求编译成交互契约图(ICG),包含可观察状态、用户意图转换以及DOM/视觉断言,以实现与实现无关的浏览器执行。WebRISE涵盖五种输入模态(文本、Markdown、草图、图像、视频)下的442个任务,包含5,495个转换和5,271个需求检查,将用户声明的功能与隐式的产品级约束分开。在14个MLLM中,即使最强的模型也仅达到65.6%的转换有效性和66.3%的需求覆盖率,且视觉质量不能代表行为(Qwen3.6-35B-A3B在Markdown上:V=80.8但T=15.5)。视频提供了最强的交互信号(隐式覆盖率比文本高10.6个百分点),而隐式约束仍然存在;缺陷注入表明,基于ICG的评分检测状态错误的速率是检查点评估的2-16倍。

英文摘要:

Existing benchmarks for MLLM-generated web artifacts assess interaction through local evidence and miss the requirement-induced states and transitions that determine whether a page works. We introduce WebRISE, which compiles task requirements into Interaction Contract Graphs (ICGs) of observable states, user-intent transitions, and DOM/visual assertions for implementation-agnostic browser execution. WebRISE spans 442 tasks across five input modalities (Text, Markdown, Sketch, Image, Video), with 5,495 transitions and 5,271 requirement checks that separate user-stated functions from implicit product-level constraints. Across 14 MLLMs, even the strongest model reaches only 65.6% transition validity and 66.3% requirement coverage, and visual quality is no proxy for behavior (Qwen3.6-35B-A3B on Markdown: V=80.8 yet T=15.5). Video gives the strongest interaction signal (+10.6 pp implicit coverage over Text), while implicit constraints persist; defect injection shows ICG-based scoring detects state errors at 2-16x the rate of checkpoint-style evaluation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑