MiniAppBench：评估从文本到交互式HTML响应的转变
MiniAppBench: Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants
AI总结：
本文提出MiniAppBench，首个评估原理驱动交互应用生成能力的基准，通过10M+生成数据提炼500个任务，结合MiniAppEval框架评估意图、静态和动态维度，揭示LLM在生成高质量交互应用上的不足。
摘要：
随着大型语言模型在代码生成中的快速发展，人机交互正从静态文本响应向动态交互式HTML应用演进，我们称之为MiniApps。这些应用要求模型不仅渲染视觉界面，还需构建符合现实原理的定制交互逻辑。然而，现有基准主要关注算法正确性或静态布局重建，无法捕捉新范式所需能力。为填补这一空白，我们引入MiniAppBench，首个全面评估原理驱动交互应用生成的基准。该基准源自一个拥有1000万+生成的现实应用，涵盖六个领域（如游戏、科学和工具）。此外，为解决开放性交互评估中缺乏单一地面真实的问题，我们提出MiniAppEval，一种代理评估框架。利用浏览器自动化，它执行类似人类的探索性测试，系统评估应用的三个维度：意图、静态和动态。实验表明，当前LLM在生成高质量MiniApps上仍面临显著挑战，而MiniAppEval与人类判断高度一致，为未来研究建立可靠标准。我们的主页可在miniappbench.github.io访问。
英文摘要：
With the rapid advancement of Large Language Models (LLMs) in code generation, human-AI interaction is evolving from static text responses to dynamic, interactive HTML-based applications, which we term MiniApps. These applications require models to not only render visual interfaces but also construct customized interaction logic that adheres to real-world principles. However, existing benchmarks primarily focus on algorithmic correctness or static layout reconstruction, failing to capture the capabilities required for this new paradigm. To address this gap, we introduce MiniAppBench, the first comprehensive benchmark designed to evaluate principle-driven, interactive application generation. Sourced from a real-world application with 10M+ generations, MiniAppBench distills 500 tasks across six domains (e.g., Games, Science, and Tools). Furthermore, to tackle the challenge of evaluating open-ended interactions where no single ground truth exists, we propose MiniAppEval, an agentic evaluation framework. Leveraging browser automation, it performs human-like exploratory testing to systematically assess applications across three dimensions: Intention, Static, and Dynamic. Our experiments reveal that current LLMs still face significant challenges in generating high-quality MiniApps, while MiniAppEval demonstrates high alignment with human judgment, establishing a reliable standard for future research. Our homepage is available in miniappbench.github.io.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
