AI 中文总结
该研究提出Generative Proxy方法，可从场景、意图和设备能力自动生成适配轻量化AR眼镜的基于代理的交互界面，经专家评估其生成的界面有用且可用，为AR交互提供新范式。
AI 中文摘要
在增强现实（AR）中与现实世界对象交互存在困难，尤其是当目标距离较远、杂乱或被遮挡时。这些挑战在新兴的轻量化AR眼镜上被进一步放大，这类眼镜通常缺乏双目显示或大视场，也缺乏诸如手部或眼部追踪之类的连续输入。基于代理的界面提供了一种替代方案，允许用户与物理对象的虚拟抽象进行交互，这些虚拟抽象可被重新定位、重组，并适配任务和设备。然而，设计此类界面目前是手动的，且高度依赖特定设备。我们提出Generative Proxy，一种从场景、意图和设备能力三个规范自动生成基于代理的界面的方法。我们将生成问题表述为约束合成问题，该问题首先为目标设备和任务生成有效界面，然后基于直接操作理论启发的语义和发音距离对候选界面进行排序。我们在多样化的场景、设备配置文件和用户意图上对Generative Proxy进行了验证。专家评估显示，初步证据表明生成的代理用户界面（UI）有用且可用，凸显基于代理的抽象是未来AR眼镜颇具前景的交互范式。
英文摘要
Interacting with real-world objects in AR is difficult, especially when targets are distant, cluttered, or occluded. These challenges are amplified on emerging lightweight AR glasses, which often lack binocular or large field of view on display, but also continuous inputs, such as hand or eye tracking. Proxy-based interfaces offer an alternative by allowing users to interact with virtual abstractions of physical objects that can be repositioned, reorganized, and adapted to the task and device. However, designing such interfaces is currently manual and highly device-specific. We present Generative Proxy, a method for automatically generating proxy-based interfaces from three specifications: scene, intent, and device capabilities. We formulate generation as a constrained synthesis problem that first produces valid interfaces for the target device and task, then ranks candidates using semantic and articulatory distance inspired by direct manipulation theory. We demonstrate Generative Proxy across diverse scenes, device profiles, and user intents. Expert evaluation shows initial evidence that generated proxy UIs are useful and usable, highlighting proxy-based abstraction as a promising interaction paradigm for future AR glasses.
Comments11 pages, 11 figures, published in UIST '26, Detroit, MI, USA, Nov 2 - 5, 2026; UIST '26: ACM Symposium on User Interface Software and Technology
