HelpViz：从基于文本的指令自动生成情境化视觉移动教程
HelpViz: Automatic Generation of Contextual Visual MobileTutorials from Text-Based Instructions
AI总结：
提出HelpViz工具，通过解析文本指令并在Android模拟器上执行动作，自动生成情境化视觉移动教程，实验证明其提高了执行鲁棒性且更受用户偏好。
摘要：
我们提出了HelpViz，这是一种从网络上大量存在的基于文本的指令生成情境化视觉移动教程的工具。HelpViz通过指令解析模型从每条文本指令中提取一系列动作，并在管理一组Android模拟器的模拟基础设施上执行提取的动作，从而批量将文本指令转换为图形教程。每条指令的自动执行会生成一组图形和结构资产，包括图像、视频以及每步点击元素等元数据。HelpViz随后将解析的文本指令与生成的资产相结合以合成教程，并通过跟踪用户进度和突出显示下一步骤使教程情境化以适应用户交互。我们使用HelpViz进行的实验表明，我们的流水线提高了教程执行的鲁棒性，且参与者更倾向于使用HelpViz生成的教程而非文本指令。HelpViz有望为大规模移动交互提供一种生成情境化视觉教程的高性价比方法。
英文摘要：
We present HelpViz, a tool for generating contextual visual mobile tutorials from text-based instructions that are abundant on the web. HelpViz transforms text instructions to graphical tutorials in batch, by extracting a sequence of actions from each text instruction through an instruction parsing model, and executing the extracted actions on a simulation infrastructure that manages an array of Android emulators. The automatic execution of each instruction produces a set of graphical and structural assets, including images, videos, and metadata such as clicked elements for each step. HelpViz then synthesizes a tutorial by combining parsed text instructions with the generated assets, and contextualizes the tutorial to user interaction by tracking the user's progress and highlighting the next step. Our experiments with HelpViz indicate that our pipeline improved tutorial execution robustness and that participants preferred tutorials generated by HelpViz over text-based instructions. HelpViz promises a cost-effective approach for generating contextual visual tutorials for mobile interaction at scale.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
