空间中的代码：多模态人机交互体验如何重塑技术创造的未来

中文摘要 AI 辅助

人工智能（AI）的进步持续重塑数字产品开发，然而开发者和设计师的日常工具仍局限于平面屏幕和二维输入。AI与扩展现实（AI-XR）的交汇引入了强大的多模态交互通道，如注视、动作或空间计算，这些可以丰富现有的人机交互体验。利用这些多模态机会的一个关键挑战在于理解如何将这些元素组合成一个连贯、高层次的创造性环境。我们的研究通过对13位AI-XR专家的半结构化访谈进行主题分析，绘制了这一领域的图谱。通过主题分析对150多个主题进行分类，我们勾勒出这一不断演变格局的五个核心维度：专业创作、作为情境层的AI、新的交互范式、采用障碍以及伦理与人的位置。我们的分析揭示，除了关键硬件限制外，AI-XR在技术创造中的未来还取决于解决人类认知极限的问题。最终，在新的多模态人机交互体验范式中取得成功，需要超越平面屏幕的隐喻，设计新型交互，这些交互有选择性地管理人类注意力，同时保护用户自主权。

英文摘要

Advances in artificial intelligence (AI) continue to reshape digital product development, yet the day-to-day tools for developers and designers remain bound to flat screens and 2D inputs. The intersection of AI and Extended Reality (AI-XR) introduces powerful multimodal interaction channels, such as gaze, motion, or spatial computing, that can enrich the existing Human-AI experience. A critical challenge for utilizing this multimodal opportunities lies in the understanding of how to combine these elements into a cohesive, high-level creative environment. Our study maps this territory through a thematic analysis of semi-structured interviews with 13 AI-XR experts. Categorizing over 150 topics through thematic analysis, we outline five core dimensions of this evolving landscape: professional creation, AI as a contextual layer, new interaction paradigms, adoption frictions, and ethics and human position. Our analysis reveals that besides the critical hardware constraints, the future of AI-XR for tech creation is dependent on addressing human cognitive limits. Ultimately, succeeding in the new multimodal Human-AI experience paradigm requires moving past flat-screen metaphors to design new types of interactions that selectively manage human attention while protecting user agency.

发表机构

• JetBrains Research（JetBrains研究）

机构由 AI 辅助整理，请以论文原文为准。