学习对原始移动UI布局进行降噪，以大规模改进数据集
Learning to Denoise Raw Mobile UI Layouts for Improving Datasets at Scale
摘要：
手机屏幕的布局是UI设计研究和屏幕语义理解的关键数据源。然而，现有数据集中的UI布局通常是嘈杂的，与其视觉表示不匹配，或者由难以分析和建模的通用或特定于应用程序的类型组成。在本文中，我们提出了CLAY管道，该管道使用深度学习方法对UI布局进行去噪，使我们能够大规模地自动改进现有的移动UI布局数据集。我们的管道同时获取屏幕截图和原始UI布局，并通过删除不正确的节点并为每个节点分配语义上有意义的类型来注释原始布局。为了试验我们的数据清理管道，我们创建了59，555个人工注释的屏幕布局的CLAY数据集，这些布局基于来自Rico的屏幕截图和原始布局，Rico是一个公共移动UI语料库。我们的深度模型实现了高准确性，F1得分为82.7%，用于检测没有有效视觉表示的布局对象，识别对象类型为85.9%，这明显优于启发式基线。我们的工作为数据驱动的移动UI研究创建大规模高质量的UI布局数据集奠定了基础，并减少了昂贵的手动标记工作的需要。
英文摘要：
The layout of a mobile screen is a critical data source for UI design research and semantic understanding of the screen. However, UI layouts in existing datasets are often noisy, have mismatches with their visual representation, or consists of generic or app-specific types that are difficult to analyze and model. In this paper, we propose the CLAY pipeline that uses a deep learning approach for denoising UI layouts, allowing us to automatically improve existing mobile UI layout datasets at scale. Our pipeline takes both the screenshot and the raw UI layout, and annotates the raw layout by removing incorrect nodes and assigning a semantically meaningful type to each node. To experiment with our data-cleaning pipeline, we create the CLAY dataset of 59,555 human-annotated screen layouts, based on screenshots and raw layouts from Rico, a public mobile UI corpus. Our deep models achieve high accuracy with F1 scores of 82.7% for detecting layout objects that do not have a valid visual representation and 85.9% for recognizing object types, which significantly outperforms a heuristic baseline. Our work lays a foundation for creating large-scale high quality UI layout datasets for data-driven mobile UI research and reduces the need of manual labeling efforts that are prohibitively expensive.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
