PlanViz: 评估面向计算机使用任务的图像生成与编辑
PlanViz: Evaluating Planning-Oriented Image Generation and Editing for Computer-Use Tasks
AI总结：
PlanViz旨在评估图像生成与编辑在计算机使用任务中的表现，通过设计日常生活中常见的子任务，如路线规划、工作图示和网页与UI显示，提出任务自适应评分体系PlanScore以评估生成图像的正确性、视觉质量和效率。
摘要：
统一多模态模型（UMMs）在生成自然图像和支持多模态推理方面表现出色。然而，其在支持与生活密切相关的计算机使用规划任务中的潜力仍待探索。图像生成与编辑在计算机使用任务中需要空间推理和过程理解能力，尚不清楚UMMs是否具备完成这些任务的能力。因此，我们提出了PlanViz，一个用于评估计算机使用任务图像生成与编辑的新基准。为实现评估目标，我们专注于日常生活中频繁出现且需要规划的子任务。具体设计了三个代表性子任务：路线规划、工作图示和网页与UI显示。通过整理人工标注的问题和参考图像，并实施质量控制流程，解决数据质量挑战。为实现详细和精确的评估，提出了任务自适应评分PlanScore，该评分有助于理解生成图像的正确性、视觉质量和效率。通过实验，我们突显了该领域未来研究的关键限制和机遇。
英文摘要：
Unified multimodal models (UMMs) have shown impressive capabilities in generating natural images and supporting multimodal reasoning. However, their potential in supporting computer-use planning tasks, which are closely related to our lives, remain underexplored. Image generation and editing in computer-use tasks require capabilities like spatial reasoning and procedural understanding, and it is still unknown whether UMMs have these capabilities to finish these tasks or not. Therefore, we propose PlanViz, a new benchmark designed to evaluate image generation and editing for computer-use tasks. To achieve the goal of our evaluation, we focus on sub-tasks which frequently involve in daily life and require planning. Specifically, three representative sub-tasks are designed: route planning, work diagramming, and web&UI displaying. We address challenges in data quality ensuring by curating human-annotated questions and reference images, and a quality control process. For detailed and exact evaluation, a task-adaptive score, PlanScore, is proposed. The score helps understanding the correctness, visual quality and efficiency of generated images. Through experiments, we highlight key limitations and opportunities for future research on this topic.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
