A11y-CUA数据集：表征计算机使用代理中的可访问性差距
A11y-CUA Dataset: Characterizing the Accessibility Gap in Computer Use Agents
AI总结：
A11y-CUA数据集通过分析BLVUs与SUs的交互差异，揭示CUAs在辅助技术条件下的性能下降，旨在推动更包容的CUA发展。
摘要：
计算机使用代理（CUAs）通过指、点击和输入操作界面，模拟了视力正常用户（SUs）的交互方式，因此SUs可以监控CUAs并共享控制权。而盲人和低视力用户（BLVUs）无法通过CUAs进行交互，因为他们使用辅助技术（AT）进行交互。为了表征CUAs的可访问性差距，我们提出了A11y-CUA数据集，该数据集记录了BLVUs和SUs完成60项日常任务，共计40.4小时和158,325个事件。我们的数据集分析揭示，所收集的交互轨迹定量证实了SU和BLVU组之间不同的交互方式（鼠标为主 vs. 键盘为主）以及组内交互的多样性（BLVUs的顺序导航 vs. 快捷键导航）。随后，我们将收集的轨迹与最先进的CUAs在默认和AT条件下进行比较（仅键盘、放大镜）。默认CUA成功完成了78.3%的任务。但使用AT条件时，CUA的性能分别降至41.67%和28.3%，且未能反映真实AT使用中的细微差别。通过我们的公开A11y-CUA数据集，我们旨在促进更协作和可访问的CUAs。
英文摘要：
Computer Use Agents (CUAs) operate interfaces by pointing, clicking, and typing -- mirroring interactions of sighted users (SUs) who can thus monitor CUAs and share control. CUAs do not reflect interactions by blind and low-vision users (BLVUs) who use assistive technology (AT). BLVUs thus cannot easily collaborate with CUAs. To characterize the accessibility gap of CUAs, we present A11y-CUA, a dataset of BLVUs and SUs performing 60 everyday tasks with 40.4 hours and 158,325 events. Our dataset analysis reveals that our collected interaction traces quantitatively confirm distinct interaction styles between SU and BLVU groups (mouse- vs. keyboard-dominant) and demonstrate interaction diversity within each group (sequential vs. shortcut navigation for BLVUs). We then compare collected traces to state-of-the-art CUAs under default and AT conditions (keyboard-only, magnifier). The default CUA executed 78.3% of tasks successfully. But with the AT conditions, CUA's performance dropped to 41.67% and 28.3% with keyboard-only and magnifier conditions respectively, and did not reflect nuances of real AT use. With our open A11y-CUA dataset, we aim to promote collaborative and accessible CUAs for everyone.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
