StateLens：使现有动态触摸屏可折叠的逆向工程解决方案
StateLens: A Reverse Engineering Solution for Making Existing Dynamic Touchscreens Accessible
摘要：
盲人在日常生活中经常遇到难以接近的动态触摸屏，这些触摸屏很困难，令人沮丧，并且通常不可能独立使用。触摸屏通常是控制从咖啡机和支付终端到地铁售票机和机上娱乐系统的唯一方式。与动态触摸屏进行非视觉交互是困难的，因为视觉用户界面会改变，交互通常发生在多个不同的屏幕上，并且在探索屏幕时很容易意外触发界面动作。为了解决这些问题，我们引入了StateLens -一个由三部分组成的逆向工程解决方案，使现有的动态触摸屏可访问。首先，StateLens使用在线找到的或用户使用混合人群-计算机视觉管道拍摄的视点视频对现有界面的底层状态图进行逆向工程。其次，使用状态图，StateLens自动生成会话代理，通过指定界面可以执行的任务来指导盲人用户，允许StateLens iOS应用程序提供交互式指导和反馈，以便盲人用户可以访问界面。最后，一组3D打印配件使盲人能够探索电容式触摸屏，而不会引发意外触摸界面的风险。我们的技术评估表明，StateLens可以准确地从固定，手持和网络视频中重建界面;并且，对整个系统的用户研究表明，StateLens成功地使盲人用户能够访问否则无法访问的动态触摸屏。
英文摘要：
Blind people frequently encounter inaccessible dynamic touchscreens in their everyday lives that are difficult, frustrating, and often impossible to use independently. Touchscreens are often the only way to control everything from coffee machines and payment terminals, to subway ticket machines and in-flight entertainment systems. Interacting with dynamic touchscreens is difficult non-visually because the visual user interfaces change, interactions often occur over multiple different screens, and it is easy to accidentally trigger interface actions while exploring the screen. To solve these problems, we introduce StateLens - a three-part reverse engineering solution that makes existing dynamic touchscreens accessible. First, StateLens reverse engineers the underlying state diagrams of existing interfaces using point-of-view videos found online or taken by users using a hybrid crowd-computer vision pipeline. Second, using the state diagrams, StateLens automatically generates conversational agents to guide blind users through specifying the tasks that the interface can perform, allowing the StateLens iOS application to provide interactive guidance and feedback so that blind users can access the interface. Finally, a set of 3D-printed accessories enable blind people to explore capacitive touchscreens without the risk of triggering accidental touches on the interface. Our technical evaluation shows that StateLens can accurately reconstruct interfaces from stationary, hand-held, and web videos; and, a user study of the complete system demonstrates that StateLens successfully enables blind users to access otherwise inaccessible dynamic touchscreens.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
