arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.03486cs.HCcs.AIcs.CVcs.LGcs.MA
收藏

VisionClaw:通过智能眼镜实现始终在线的AI代理

VisionClaw: Always-On AI Agents through Smart Glasses

University of Colorado Boulder(科罗拉多大学博尔德分校);Gwangju Institute of Science and Technology(光州科学技术院);Google(谷歌)

Xiaoan Liu, DaeHo Lee, Eric J Gonzalez, Mar Gonzalez-Franco, Ryo Suzuki

AI总结:

VisionClaw通过智能眼镜实现始终在线的AI代理,结合实时第一视角感知与任务执行,支持语音驱动的任务启动与委托,提升任务完成效率和交互体验。

摘要:

我们提出了VisionClaw,一种始终在线的可穿戴AI代理,它集成了实时第一视角感知与代理任务执行。该系统基于Meta Ray-Ban智能眼镜运行,能够持续感知现实世界环境,并通过OpenClaw AI代理实现现场语音驱动的任务启动和委托。因此,用户可以通过智能眼镜直接执行任务,例如将现实中的物体添加到亚马逊购物车、从物理文档生成笔记、在移动中接收会议简报、从海报创建事件或控制物联网设备。我们通过受控实验室研究(N=12)和纵向部署研究(N=5)评估了VisionClaw。结果表明,整合感知与执行能够加快任务完成速度,并减少与非始终在线和非代理基线相比的交互开销。除了性能提升外,部署发现揭示了交互模式的转变:任务在持续活动中机会性启动,执行越来越多地委托而非手动控制。这些结果表明了一种新的可穿戴AI代理范式,其中感知和行动持续耦合以支持现场、无手交互。

英文摘要:

We present VisionClaw, an always-on wearable AI agent that integrates live egocentric perception with agentic task execution. Running on Meta Ray-Ban smart glasses, VisionClaw continuously perceives real-world context and enables in-situ, speech-driven action initiation and delegation via OpenClaw AI agents. Therefore, users can directly execute tasks through the smart glasses, such as adding real-world objects to an Amazon cart, generating notes from physical documents, receiving meeting briefings on the go, creating events from posters, or controlling IoT devices. We evaluate VisionClaw through a controlled laboratory study (N=12) and a longitudinal deployment study (N=5). Results show that integrating perception and execution enables faster task completion and reduces interaction overhead compared to non-always-on and non-agent baselines. Beyond performance gains, deployment findings reveal a shift in interaction: tasks are initiated opportunistically during ongoing activities, and execution is increasingly delegated rather than manually controlled. These results suggest a new paradigm for wearable AI agents, where perception and action are continuously coupled to support situated, hands-free interaction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑