arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.03507cs.CVcs.SE
收藏

使用SOTA YOLO深度学习模型进行GUI元素检测

GUI Element Detection Using SOTA YOLO Deep Learning Models

University of Manitoba(曼尼托巴大学)

Seyed Shayan Daneshvar, Shaowei Wang

AI总结:

本文针对GUI元素检测这一领域特定任务,系统评估了四种最新SOTA YOLO模型,揭示通用目标检测模型的比较结论可能不适用于GUI场景,并考察了各模型在检测不同GUI元素时的准确率表现。

摘要:

图形用户界面(GUI)元素的检测是从图像和草图自动生成代码、GUI测试以及GUI搜索中的一项关键任务。近期研究同时利用了传统和现代的计算机视觉(CV)技术。传统方法使用经典图像处理算法(例如边缘检测和轮廓检测),而现代方法则采用成熟的深度学习方案来处理通用目标检测任务。然而,GUI元素检测是目标检测的一个领域特定场景,在该场景中,目标之间重叠更为频繁,彼此位置非常接近,同时目标类别数量明显更少,但与自然图像相比,图像中的目标数量却更多。因此,此前针对各种目标检测模型所进行的比较研究,可能并不适用于GUI元素检测。在本研究中,我们评估了四种最新的、在通用目标检测任务中表现优异的YOLO模型在GUI元素检测上的性能,并考察了它们在检测各类GUI元素时的准确率表现。

英文摘要:

Detection of Graphical User Interface (GUI) elements is a crucial task for automatic code generation from images and sketches, GUI testing, and GUI search. Recent studies have leveraged both old-fashioned and modern computer vision (CV) techniques. Oldfashioned methods utilize classic image processing algorithms (e.g. edge detection and contour detection) and modern methods use mature deep learning solutions for general object detection tasks. GUI element detection, however, is a domain-specific case of object detection, in which objects overlap more often, and are located very close to each other, plus the number of object classes is considerably lower, yet there are more objects in the images compared to natural images. Hence, the studies that have been carried out on comparing various object detection models, might not apply to GUI element detection. In this study, we evaluate the performance of the four most recent successful YOLO models for general object detection tasks on GUI element detection and investigate their accuracy performance in detecting various GUI elements.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑