应用意识:使用计算机视觉和密码学进行测试驱动的GUI开发
Applied Awareness: Test-Driven GUI Development using Computer Vision and Cryptography
摘要:
图形化用户界面测试是一项非常具有挑战性的工作,而自动化测试更是如此。测试驱动的开发是不切实际的:它通常需要GUI的初始实现来生成黄金映像或构建交互式测试场景,并且后续的维护成本很高。虽然计算机视觉已被应用到GUI测试的几个方面,我们展示了一种新的和立即适用的方法来解释GUI演示文稿的后端通信,建模“意识”的方式采用加密的安全证明。这种对后端通信的关注避免了依赖于平台相关UI启示或可访问性特性的典型测试方法的缺陷。我们的跨学科工作已经为现成的实践做好了准备:我们报告了在线和离线验证的独立实用的实现,在一开始就使用简单的设计师规范,特别是避免了对引导实现或黄金映像的任何要求。除了实际的实现,在密码学中的形式化验证方法的关系进行了探索和解释,在UI和AI的可解释性的保证提供肥沃的前景。
英文摘要:
Graphical user interface testing is significantly challenging, and automating it even more so. Test-driven development is impractical: it generally requires an initial implementation of the GUI to generate golden images or to construct interactive test scenarios, and subsequent maintenance is costly. While computer vision has been applied to several aspects of GUI testing, we demonstrate a novel and immediately applicable approach of interpreting GUI presentation in terms of backend communications, modeling "awareness" in the fashion employed by cryptographic proofs of security. This focus on backend communication circumvents deficiencies in typical testing methodologies that rely on platform-dependent UI affordances or accessibility features. Our interdisciplinary work is ready for off-the-shelf practice: we report self-contained, practical implementation with both online and offline validation, using simple designer specifications at the outset and specifically avoiding any requirements for a bootstrap implementation or golden images. In addition to practical implementation, ties to formal verification methods in cryptography are explored and explained, providing fertile perspectives on assurance in UI and interpretability in AI.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。