发表机构
Centre for Artificial Intelligence, University College London（伦敦大学学院人工智能中心）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出AMUSER多模态框架，利用LLM智能体模拟用户行为自动生成UX改进建议，在商业网站上以89%更低的成本显著优于纯文本模拟，并揭示了多模态在模拟与生成阶段的不对称作用。
AI 中文摘要
通过用户测试评估在线网站的用户体验（UX）成本高昂、主观性强且难以规模化。LLM智能体通过模拟真实用户行为，为自动化UX测试提供了一条有前景的途径。然而，现有的模拟方法通常缺乏多模态能力，并且需要耗时的人工审查来提取可操作的见解。我们将从模拟数据中生成UX改进建议形式化为一个结构化的自然语言生成与排序问题，并建立了一个使用专家标注和LLM-as-a-Judge的评估协议。我们提出了AMUSER，一个多模态框架，它模拟用户行为并自动从生成的数据中产生优先排序的UX改进建议。我们在商业网站上评估了AMUSER，结果表明其建议在模拟成本降低89%的情况下，显著优于纯文本模拟（NDCG@3 = 0.758对比0.359）。我们的结果揭示了多模态的不对称作用：模拟期间的可视化访问通过更丰富的轨迹改进了建议，而在建议生成期间提供视觉输入可能会适度降低质量。我们还讨论了将AMUSER应用于商业网站的实际部署经验。
英文摘要
Evaluating user experience (UX) on live websites through user testing is expensive, subjective, and difficult to scale. LLM agents offer a promising route to automating UX testing by simulating realistic user behaviour. However, existing simulation approaches typically lack multimodality and require time-consuming manual review to extract actionable insights. We formalise UX improvement recommendation from simulation data as a structured natural language generation and ranking problem, and establish an evaluation protocol using expert annotation and LLM-as-a-Judge. We present AMUSER, a multimodal framework which simulates user behaviour and automatically generates prioritised UX improvement recommendations from resulting data. We evaluate AMUSER on commercial websites and show that its recommendations substantially outperform those from text-only simulation (NDCG@3 = 0.758 versus 0.359) at an 89% lower simulation cost. Our results suggest an asymmetric role of multimodality: visual access during simulation improves recommendations through richer traces, while providing visual inputs during recommendation generation can modestly degrade quality. We also discuss practical deployment lessons from applying AMUSER to commercial websites.
