arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2405.18537cs.HCcs.LG
收藏

增强现实中的嵌入式语音驱动即时引用增强对话

Augmented Conversation with Embedded Speech-Driven On-the-Fly Referencing in AR

University of Calgary(卡尔加里大学);Snap Research(Snap研究部门);Easel AI(Easel AI公司)

Shivesh Jadon, Mehrad Faridan, Edward Mah, Rajan Vaish, Wesley Willett, Ryo Suzuki

AI总结:

本文提出增强对话概念,通过AR中嵌入式语音驱动即时引用,减少干扰、保持眼神交流,并开发了基于Hololens的系统,初步研究证实其优于智能手机搜索。

摘要:

本文介绍了增强对话的概念,旨在通过增强现实(AR)中的嵌入式语音驱动即时引用,支持同地面对面对话。如今,智能手机等计算技术允许在对话期间快速访问各种参考资料。然而,这些工具常常造成干扰,减少眼神交流,并迫使用户将注意力集中在手机屏幕上,手动输入关键词以获取相关信息。相比之下,基于AR的即时引用根据从口语对话中自动提取的关键词,实时提供相关的视觉参考。通过将这些视觉参考嵌入到对话伙伴周围的AR中,增强对话减少了干扰和摩擦,使用户能够保持眼神交流,并支持更自然的社交互动。为演示这一概念,我们开发了\tsystem,一个基于Hololens的界面,利用实时语音识别、自然语言处理和基于凝视的交互,实现即时嵌入式视觉引用。在本文中,我们探索了对话中视觉引用的设计空间,并描述了我们的实现——基于通过以用户为中心的设计过程确定的七项设计指南。一项初步用户研究证实,与智能手机搜索相比,我们的系统减少了对话中的干扰和摩擦,同时提供了高度有用和相关的信息。

英文摘要:

This paper introduces the concept of augmented conversation, which aims to support co-located in-person conversations via embedded speech-driven on-the-fly referencing in augmented reality (AR). Today computing technologies like smartphones allow quick access to a variety of references during the conversation. However, these tools often create distractions, reducing eye contact and forcing users to focus their attention on phone screens and manually enter keywords to access relevant information. In contrast, AR-based on-the-fly referencing provides relevant visual references in real-time, based on keywords extracted automatically from the spoken conversation. By embedding these visual references in AR around the conversation partner, augmented conversation reduces distraction and friction, allowing users to maintain eye contact and supporting more natural social interactions. To demonstrate this concept, we developed \system, a Hololens-based interface that leverages real-time speech recognition, natural language processing and gaze-based interactions for on-the-fly embedded visual referencing. In this paper, we explore the design space of visual referencing for conversations, and describe our our implementation -- building on seven design guidelines identified through a user-centered design process. An initial user study confirms that our system decreases distraction and friction in conversations compared to smartphone searches, while providing highly useful and relevant information.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑